Integrate Systems with a Transform
In this tutorial you'll learn how to:
- Design a Multi-Transport System
- Integrate different Message Types with a Transform
- Generate compiled Interfaces, Transform Code and Transport Libraries
- Create Containers and run an End-to-End Transform Test inside them, all within Tangram Pro!
Scenario
You've just started as an Integration Engineer at SkyShield, a small startup in the electronic warfare space serving the Air Force. Your team at SkyShield is working with RadarWarfare Solutions (the "prime") on a follow-on project to demonstrate their platform integrating with SateSphere's platform.
The companies represented here are for illustrative purposes and are not real
RadarWarfare's systems communicate using the ZeroMQ messaging library and the OpenUxAS LMCP message standard, while SateSphere's systems communicate over ActiveMQ and use the NATO STANAG4586 message standard.
Your team is evaluating the effectiveness of an integration tool like Tangram Pro to help them integrate and test their systems. Your team has defined a Flex Transform in Tangram Pro to convert the LMCP AirVehicleState
message type to the STANAG4586 InertialStates
message type. Now you're ready to build and test the Transform inside Tangram Pro in a Multiple Transport scenario.
Design the System
1. Create a Project
To begin modeling a system we need to make a project. Projects are used to design component-based systems, generate and run code based on your design. Let's get started!
- Go to Projects and click New Project
- Enter a Project Name
- You can associate a team to your project with the Select Owner dropdown menu, or keep Personal (You) selected
2. Add Transports
The model of our integration scenario requires two different messaging libraries, or what we call Transports. This is easy to design in Tangram Pro. Let's start by adding ZeroMQ and ActiveMQ.
- Click the Transports button and add ZeroMQ
- Repeat and add ActiveMQ
All the Transports in your design are listed in the Navigator on the left. We'll make use of these when we start connecting components. The one with the asterisk (*) designates the default transport. For more information see User Guide: Working with Transports
3. Add Components
Add a Component to represent a RadarWarfare Solutions software component that sends the AirVehicleState LMCP message type
- Add a Component and name it
RWS_LMCP
- Go to the Interface tab and choose:
- Message API:
Tangram Pro Default API
- Package:
OpenUxAS::LMCP::v3
- Message:
AirVehicleState
and selectOutput
- Message API:
Add a Component to represent a SateSphere software component that receives the InertialStates STANAG4586 message type
- Add a Component and name it
SS_STANAG
- Go to the Interface tab and choose
- Message API:
Tangram Pro Default API
- Package:
NATO::STANAG4586::v3
- Message:
InertialStates
and selectInput
- Message API:
4. Add a Transform
A Transform allows you to integrate different message types in Tangram Pro, and not just visually in the design. With the click of a button, Tangram Pro will generate an executable Transform application based on your model with safeguarded input and output interfaces (e.g. the message types allowed, serializer/deserializer needed and transport to use).
For our integration scenario, we'll use a Transform to relate the LMCP AirVehicleState
and STANAG4586 InertialStates
message types. Both of these messages contain information about the position and velocity of an aircraft and are used for similar tasks, but they are structured differently and comprised of data types that are specific to the LMCP and STANAG4586 format standards.
Tangram Pro uses FlexLang, a specification language, to describe Transforms. Flex enables accurate and readable transform specifications that can be transpiled into a variety of languages and output formats. We're going to use an existing AirVehicleState to InertialStates Transform for this tutorial, but you can always author your own.
- Click the Add Transform button and place it between your 2 Components
- Define the direction of message flow by first clicking on the
RWS_LMCP
Component, then theSS_STANAG
Component - Set the Transform to input messages with ZeroMQ and output messages with ActiveMQ
- Use the Suggested button to add your Messages
- Input Message:
- ZeroMQ
- OpenUxAS::LMCP::v3
- AirVehicleState
- Output Message:
- ActiveMQ
- NATO::STANAG4586::v3
- InertialStates
- Input Message:
- Choose the
AVS2IS
Transform. Your Transform settings are automatically saved. You can click in an empty part of the Design Stage to close the Transform panel.
Now that your Transform is added, try hovering over each Transport listed in the Navigator. The connections that are using the Transport will be highlighted in the design.
Configure Build
Nice work! You've just created a model of the integration scenario. From this model, Tangram Pro can generate code, compile and place it in containers to test. Your Project Design acts as the source of truth for your system interfaces (you've just specified all of the interface details as you created the model) so generating code is easy and repeatable.
In this section, you'll configure Tangram Pro to:
- Generate LMCP Interface Code, which will serialize & send AirVehicleState messages to ZeroMQ
- Generate STANAG4586 Interface Code, which will deserialize & receive InertialStates messages from ActiveMQ
- Generate a Transform application, which will deserialize incoming AirVehicleState messages from ZeroMQ, transform the data, and serialize & send the resulting InertialStates messages to ActiveMQ
- Containerize the entire system. All Tangram Pro needs is a Dockerfile, and it will create a container image and push that image to the Tangram Pro container registry. You can run these containers to test your system.
We're also going to utilize end-to-end tests that are generated along side the interfaces and transform code.
5. LMCP Build Actions
First we'll configure Build Actions for the LMCP Component. We want to generate Interface Code for LMCP that we can run in a container. Specifically, we want to run the generated end-to-end test application named "Writer". In the Dockerfile provided below you'll see that we set the Transport to zeromq
, move into test/end-to-end
and run ./afrl_cmasi_AirVehicleStateWriter
in its container.
- Click Build in the top left menu
- Select the
RWS_LMCP
Component - Go to the Actions tab and enable:
- Generate Interface Code: C++
- Compile Code: all
- Containerize
- Include Generated Code
- Use Dockerfile Upload
- Copy the code provided below into a plain text file named "Dockerfile"
- Upload Dockerfile
Click to view Dockerfile code
docker
FROM tangramflex/pro-builder:latest AS buildARG package=lmcpARG transport=zeromqARG transport_hostname=zeromq-transportARG binary=afrl_cmasi_AirVehicleStateWriterENV transport_hostname=${transport_hostname}ENV package=${package}ENV binary=${binary}COPY code-gen/local_install /root/local_installCOPY code-gen/pkg_deps /root/pkg_depsCOPY code-gen/test /root/testCOPY code-gen/include /root/includeCOPY code-gen/build /root/buildRUN sed -i "s/127.0.0.1/${transport_hostname}/g" /root/pkg_deps/genericapi/pkg_deps/transports-cpp/transports.configRUN cd /root/test/end-to-end && make -j4 serializer=${package}ENV LD_LIBRARY_PATH=/root/local_install/lib:/root/build/libsWORKDIR /root/test/end-to-endCMD [ "sh", "-c", "sleep 60 && ./${binary}" ]
6. STANAG Build Actions
Next, We want a Interface Code for STANAG4586 that we can run in a container. Specifically, we want to run the generated end-to-end test application named "Reader". In the Dockerfile provided below you'll see that we set the Transport to activemq
, move into test/end-to-end
and run ./Reader
in its container.
- Select the
SS_STANAG
Component - Go to the Actions tab and enable:
- Generate Interface Code: C++
- Compile Code: all
- Containerize
- Include Generated Code
- Use Dockerfile Upload
- Copy the code provided below into a plain text file named "Dockerfile"
- Upload Dockerfile
Click to view Dockerfile code
docker
FROM tangramflex/pro-builder:latest AS buildARG package=stanag4586ARG transport=activemqARG transport_hostname=activemq-transportARG binary=ReaderENV transport_hostname=${transport_hostname}ENV package=${package}ENV binary=${binary}COPY code-gen/local_install /root/local_installCOPY code-gen/pkg_deps /root/pkg_depsCOPY code-gen/test /root/testCOPY code-gen/include /root/includeCOPY code-gen/build /root/buildRUN sed -i "s/127.0.0.1/${transport_hostname}/g" /root/pkg_deps/genericapi/pkg_deps/transports-cpp/transports.configRUN cd /root/test/end-to-end && make -j4 serializer=${package}ENV LD_LIBRARY_PATH=/root/local_install/lib:/root/build/libsWORKDIR /root/test/end-to-endCMD [ "sh", "-c", "sleep 3 && ./${binary}" ]
7. Transform Build Actions
Lastly, we want a fully built Transform application that we can run in a container. In the Dockerfile provided below you'll see that we set the to and from properties accordingly, and run ./transform
in its container.
- Select the Transform
- Go to the Actions tab
- Choose the option Generate Transform with CSIs
- Enable Containerize and remove the default file by clicking Remove Dockerfile
- Copy the code provided below into a plain text file named "Dockerfile"
- Upload Dockerfile
Click to view Dockerfile code
docker
FROM tangramflex/pro-builder:latest AS buildARG from_package=lmcpARG to_package=stanag4586ARG from_transport=zeromqARG to_transport=activemqARG from_transport_hostname=zeromq-transportARG to_transport_hostname=activemq-transportENV from_package=${from_package}ENV to_package=${to_package}ENV from_transport=${from_transport}ENV from_transport_hostname=${from_transport_hostname}ENV to_transport=${to_transport}ENV to_transport_hostname=${to_transport_hostname}# copy the generated transform code into the containerCOPY out/transform /root/transformCOPY ./transform/${provider} /root/transform/COPY out/${from_package} /root/${from_package}COPY out/${to_package} /root/${to_package}# disable contextual param errorRUN if [ 0 -lt $(ls /root/transform/Provider_*.hpp 2>/dev/null | wc -w) ]; then sed -i '/stderr/d' /root/transform/Provider_*.hpp; fiRUN if [ 0 -lt $(ls /root/transform/Provider_*.hpp 2>/dev/null | wc -w) ]; then sed -i '/abort/d' /root/transform/Provider_*.hpp; fi# rebuild app to apply changesRUN make -C "/root/transform" cleanRUN make -C "/root/transform" in_transport=$from_transport out_transport=$to_transport# hardcoded transport nameRUN sed -i "s/127.0.0.1/${from_transport_hostname}/g" /root/transform/transport_${from_transport}_in.configRUN sed -i "s/127.0.0.1/${to_transport_hostname}/g" /root/transform/transport_${to_transport}_out.configWORKDIR /root/transformENV LD_LIBRARY_PATH=/root/${from_package}/local_install/lib/:/root/${from_package}/build/libs/:/root/${to_package}/local_install/lib/:/root/${to_package}/build/libs/CMD [ "sh", "-c", "sleep 2 && ./transform" ]
Run End-to-End Test
Great! Now you're all set and ready to generate code and test it out. In this section you'll:
- Complete a Build, which generates code and containers for the entire system
- Run an End-to-End Test in Containers, which will verify that the generated interfaces and transform are integrated
8. Generate Code
- Click the blue Build button
- Grab a cup of coffee ☕ and check out the streaming build logs. This may take up to 30 minutes to complete the first time as it is compiling the component interfaces, transform application and two transport libraries. Subsequent builds will use cached artifacts and be much quicker
You can close Builds that are in progress at any time and return to the Design mode, or work on another project. A notification will pop up in Tangram Pro when the Build is complete. Click Builds on the top right menu to access all previous Builds.
9. Run Containers
Downloadable artifacts are available for completed Builds if you wish to run the code locally. Since we provided Dockerfiles, let's avoid that hassle and run the code inside Tangram Pro containers!
- Select Run in the top left menu after the Build completes
- Select the completed Build and click the Run button
- Click the Run button
Tangram Pro will spin up the container images created during the build, and display each container's execution via system logs. You'll see 5 containers spin up and begin executing their commands. It may take a minute before they're all running.
- ZeroMQ starts a proxy service
- ActiveMQ starts a proxy service
- LMCP Component executes the end-to-end test app "Writer" which sends a serialized AirVehicleState message to ZeroMQ and logs the message contents
- Transform executes the transform app which listens to ZeroMQ for the AirVehicleState message, deserializes it, transforms it, serializes it in the STANAG format and sends it to ActiveMQ
- STANAG Component executes the end-to-end test app "Reader" which listens for a serialized InertialStates message from ActiveMQ and logs the received message contents
If the STANAG container logs a message, then the integration was successful!
Cheers!
Your team has successfully completed the Integration Challenge using Tangram Pro 🎉 By completing this tutorial, you've learned how to integrate two systems that use different message standards, and messaging services (or Transports.)
Check out what you accomplished:
- Designed a Multi-Transport System
- Integrated different Message Types with a Transform
- Generated compiled Interfaces, Transform Code and Transport Libraries
- Created Containers and ran an End-to-End Transform Test inside them, all within Tangram Pro!
Additional User Guide Resources:
If you want to know more about component-based system design in Tangram Pro, take a look at our blog posts.