How to pass array using FluidEarth
I am starting to wrap models using FluidEarth. Now I understand how to pass single values between models, but not sure how to pass array (such as 2D spatial distributed groundwater levels) between models using FluidEarth. Is there any examples for this?
I've quickly put together a simple model that increments the values held in a 2D array; I have attached this model to my reply. This model can pass the values it has calculated (from its initialised variables) to another model, or accept a compatible 2D array of data from a second model (note: this model assumes that the data is already in the correct format for it to use - if this isn't the case, the data would need to be interpolated before being passed-in). I have provided a composition in the .\Test2D\bin\data\ folder that uses two of these models, exchanging data between them. It is probably worth running this composition first, and checking the output files each model produces to see what is happening with the composition.
In terms of how to exchange a 2D array of data, I would suggest you first look at the .omi file; you will see I am setting up the X- and Y-size of the array the model will use. If you then look in the .xml file, you will see that I am setting up an input (or output) exchange item on a regular grid that creates an array of data to transfer that is exactly the same size.
The imoprtant part of the code is in 'Get' and 'Set'; here, I map the 2D array of data into a 1D exchange array, and back again. This is because the data is exchanged between fluidearth models as a 1D array. You will need to do something similar for your model, so it is worth understanding what is happening here.
If you wish to exchange data with models on a different grid/mesh, you may also need to export the coordinates for which you require data. This way, you can set-up an interpolator (i can upload one for you, if you need) to extract data at those points where your model requires it.
hope this helps,
Many thanks for your very quick and useful reply. I am using C# to wrap models, but the xml and omi file would give me a clue how to do this. At the moment, the format and spatial resolution of 2D array are the same between models. If I have any problem in doing this, I will get back to you.
No problem - if you feel a C# example would be useful, I can put one together for you later in the week. Really, the important concept is of mapping a 2D array into 1D for the exchange, and making sure you have described an element set in the .xml that's the correct size to hold all your data. Hopefully this will get you started, but if not let me know.
1. When try to run the pipistrelle composition you sent to me for Test2D. I got the error message "Unable to load DLL 'Test2DEngine.dll': The specified module could not be found (Exception from HRESULT: 0x8007007E)" in the log file.
2. Since this is the first time for am to use FluidEarth (although I did the training course long time ago), I think I still do not get how to pass the array between models. I created a C# project named Test2D (as attached zip file) which has a problem for inputting and outputting array. Could you please help me to check my C# files. Thanks.
3. We are trying to link groundwater model (ZOOMQ3D), a post-processing model, and two other economic models to test the concept of OpenMI. Between post-processing model and economic models, the distributed groundwater could be exchanged using XYPoint array (with same spatial resolution and same range). Now, there is another problem. Chris told me that ZOOMQ3D_OPENMI is exporting "XYPolygon" data. Could you please give me some suggestions what is the best way to exchange groundwater levels between ZOOMQ3D and the post-processing model that uses 2D array?
I have just downloaded the model I attached to my reply, and it definitely contains test2dengine.dll mentioned in your message. This means there is a a problem with dependencies on your machine - I guess you do not have the Intel Fortran compiler installed? Attached to this reply is a zip with two dlls that you should place in the Test2D\bin\ folder and then try to re-run the composition. With these two dlls present in the same folder as test2dengine.dll, it should run correctly.
Unfortunately, I haven't been able to download your attachment - I have struggled to download zip files from sharepoint websites in the past, so have always changed the extension to something else (.RENAME is my preference). Could you rename the file, and re-uploading it?
From the sounds of the model you are looking to couple with, I guess we might need to use a slightly different element set to the regular grid used in my example. Not a problem, but it would help if I could take a look at the .xml file of the model you are interested in coupling with - could you include that in any remaned zip file you upload as a reply? In the mean time, i will put together a simple 2D cs example for you and post it on the forum a little later in the day.
all the best,
Attached a 2D C# example; this should hopefully give you an idea about how to work with 2D arrays of data and also how to call existing code from a wrapper. It's pretty simple, but it should give you an idea of what you need to do with your own models. We can take a look at creating different exchange items if you upload a copy of the .xml file of the model to which you wish to couple.
let me know if any problems,
I much appreciate your quick and very useful help and your time.
The composition is running correctly now. You are right I do not have Intel Fortran compiler installed on my PC.
Since you have create a new C# code for this, I think I don't have to send you mine. I will look at your C# sample and find out what the problems my code has.
ZOOMQ3D was wrapped using OpenMI instead of FluidEarth, so there is no xml file for ZOOMQ3D model. The element sets were defined in wrapper. attached "C#lines_to_prepare_XYPolygon_ElementSets_ZOOMQ3D.dat" file is the definition of XYPolygon for the model (Chris said that the c# wrapper can be download from Portal); "ZOOMQ3D.png" is the extent of XYPolygon in the study area displayed in Pipistrelle. I have tested that a model can get correct groundwater level for a XYPoint in Pipistrelle; "MABSWEC.omi" is the simple omi file used for ZOOM model.
Thanks again for your time and help!
no problem. glad you've got the composition running - a real benefit of openmi is that you are able to couple together models written in different languages.
ok. if you get time, take a look at the c# example and make sure that you are comfortable with how you will need to create/use data in a 1D array when it is moved between 2D models. as you will be using polygons rather than a regular grid of points, there will be some differences, but you will still need to do the same sort of thing.
thanks for the info - sounds like you are making good progress with your tests. i will begin to rewrite my c# example this weekend to demonstrate how to use xypolygons rather than a regular grid. i will try to upload this early next week, but this will depend on how much time i can find this weekend.
What a clever way for converting between 1D and 2D arrays in your code! I think I understand now how an regular array is exchanged between models in FluidEarth.
I test my model and find that the data can now be exchanged from irregular XYPolygons in ZOOM model by using regular 2D array. However there a lot of points outside of ZOOM XYPolygons that are getting value of "0" all the time, and these useless points could affect the efficiency of the running FluidEarth model. I am wondering if there is a better way other than using regular 2D array.
I have another more urgent problem to solve at the moment: with further development of models, I start to realise that I have to use 1D array as well, I guess this is quite similar to using 2D array, but I don't know how to define 1D array in xml files. Could you please let me know where I can find the document describing the definition of different data types in FluidEarth. Thanks.
Sorry I haven't uploaded an example using XYPolygons as yet - I will try to find some time later this week. It sounds as though you are making good progress with your work, however.
In terms of transfering more data than you wish to use, I would suggest the fundamental problem here is that both models in the composition (i.e. the one exporting data and the one importing data) need to have identically-sized arrays. If your source of data covers only a sub-region of your target domain, you might need to revisit the target model (ZOOM, i guess) to create an element-set that has only those elements of interest. This way, you would only be passing relevant data between the models. You may need to rewrite the parts of the ZOOM model to make sure it can handle this type of incoming data, though, and I don't know how easy/difficult this might be.
If you are comfortable using the 2D grid method, you can set N or M set to 1 and dX/dY to 0.0 (this is still a 2D array!) - alternatively, you may prefer to do things with an element-set of polylines. I will include this in the example I will put together later in the week.
At present, the best documentation is Adrian Harper's 'Developer's Manual and Reference' document, which is installed with Pipistrelle. On my machine, it is saved here:
C:\Program Files\Fluid Earth\Pipistrelle Plugins 1.0 1.0\Model Geometry Editor\FluidEarth.pdf
It is worth pointing out that v2.0 of the SDK and Pipistrelle are in development, and will be available from the portal in the coming months - if you do not find what you are looking for here, you may need to look through the source-code for the SDK itself as it is unlikely that further documentation for v1.0 SDK will be produced.
I hope this helps,
Sorry I have to bother you again. I met a problem for exchange the mixture types of data. If the exchange data is only one type (2D array or single values), it works very well. However, when I try to exchange the mixture types of data, I got "index out of bounds of array" problem, even I was exchanging the single value data. Could you please have a look at zipped attached files.
I am wondering if there is a runtime property to identify the type of the exchange data in the functions of WriteOutputs and ReadInputs. If there is one it would be possible to selectively choose the data exchange functions for array or the single value.
I replaced the attached file
Sorry for the delay in leaving you a reply. Attached is a pdf going through what I see as two main points - the handling of mixed scalar/vector data, and the use of polygons with a model. I think these two issues are inter-related, so sorting one out should help sort out both of your problems.
To hopefully get you going again, I have rewritten my Test2DCS model and created a couple of extra compositions; this shows how to create polygons using the element set factory. It also shows how to use data operations.
I hope this helps - let me know if you have any other problems.
I really appreciate your very kind and useful help.