Integrate Systems with a Transform

In this tutorial you'll learn how to:

• Design a Multi-Transport System
• Integrate different Message Types with a Transform
• Generate compiled Interfaces, Transform Code and Transport Libraries
• Create Containers and run an End-to-End Transform Test inside them, all within Tangram Pro!

Scenario

You've just started as an Integration Engineer at SkyShield, a small startup in the electronic warfare space serving the Air Force. Your team at SkyShield is working with RadarWarfare Solutions (the "prime") on a follow-on project to demonstrate their platform integrating with SateSphere's platform.

The companies represented here are for illustrative purposes and are not real

RadarWarfare's systems communicate using the ZeroMQ messaging library and the OpenUxAS LMCP message standard, while SateSphere's systems communicate over ActiveMQ and use the NATO STANAG4586 message standard.

Your team is evaluating the effectiveness of an integration tool like Tangram Pro to help them integrate and test their systems. Your team has defined a Flex Transform in Tangram Pro to convert the LMCP AirVehicleState message type to the STANAG4586 InertialStates message type. Now you're ready to build and test the Transform inside Tangram Pro in a Multiple Transport scenario.

Design the System

1. Create a Project

To begin modeling a system we need to make a project. Projects are used to design component-based systems, generate and run code based on your design. Let's get started!

1. Go to Projects and click New Project
2. Enter a Project Name
3. You can associate a team to your project with the Select Owner dropdown menu, or keep Personal (You) selected

2. Add Transports

The model of our integration scenario requires two different messaging libraries, or what we call Transports. This is easy to design in Tangram Pro. Let's start by adding ZeroMQ and ActiveMQ.

1. Click the Transports button and add ZeroMQ
2. Repeat and add ActiveMQ

All the Transports in your design are listed in the Navigator on the left. We'll make use of these when we start connecting components. The one with the asterisk (*) designates the default transport. For more information see User Guide: Working with Transports

3. Add Components

Add a Component to represent a RadarWarfare Solutions software component that sends the AirVehicleState LMCP message type

1. Add a Component and name it RWS_LMCP
2. Go to the Interface tab and choose:
   • Message API: Tangram Pro Default API
   • Package: OpenUxAS::LMCP::v3
   • Message: AirVehicleState and select Output

Add a Component to represent a SateSphere software component that receives the InertialStates STANAG4586 message type

1. Add a Component and name it SS_STANAG
2. Go to the Interface tab and choose
   • Message API: Tangram Pro Default API
   • Package: NATO::STANAG4586::v3
   • Message: InertialStates and select Input

4. Add a Transform

What is a Transform?

A Transform allows you to integrate different message types in Tangram Pro, and not just visually in the design. With the click of a button, Tangram Pro will generate an executable Transform application based on your model with safeguarded input and output interfaces (e.g. the message types allowed, serializer/deserializer needed and transport to use).

For our integration scenario, we'll use a Transform to relate the LMCP AirVehicleState and STANAG4586 InertialStates message types. Both of these messages contain information about the position and velocity of an aircraft and are used for similar tasks, but they are structured differently and comprised of data types that are specific to the LMCP and STANAG4586 format standards.

Tangram Pro uses FlexLang, a specification language, to describe Transforms. Flex enables accurate and readable transform specifications that can be transpiled into a variety of languages and output formats. We're going to use an existing AirVehicleState to InertialStates Transform for this tutorial, but you can always author your own.

1. Click the Add Transform button and place it between your 2 Components
2. Define the direction of message flow by first clicking on the RWS_LMCP Component, then the SS_STANAG Component
3. Set the Transform to input messages with ZeroMQ and output messages with ActiveMQ
4. Use the Suggested button to add your Messages
   • Input Message:
     • ZeroMQ
     • OpenUxAS::LMCP::v3
     • AirVehicleState
   • Output Message:
     • ActiveMQ
     • NATO::STANAG4586::v3
     • InertialStates
5. Choose the AVS2IS Transform. Your Transform settings are automatically saved. You can click in an empty part of the Design Stage to close the Transform panel.

tip

Now that your Transform is added, try hovering over each Transport listed in the Navigator. The connections that are using the Transport will be highlighted in the design.

Configure Build

Nice work! You've just created a model of the integration scenario. From this model, Tangram Pro can generate code, compile and place it in containers to test. Your Project Design acts as the source of truth for your system interfaces (you've just specified all of the interface details as you created the model) so generating code is easy and repeatable.

In this section, you'll configure Tangram Pro to:

• Generate LMCP Interface Code, which will serialize & send AirVehicleState messages to ZeroMQ
• Generate STANAG4586 Interface Code, which will deserialize & receive InertialStates messages from ActiveMQ
• Generate a Transform application, which will deserialize incoming AirVehicleState messages from ZeroMQ, transform the data, and serialize & send the resulting InertialStates messages to ActiveMQ
• Containerize the entire system. All Tangram Pro needs is a Dockerfile, and it will create a container image and push that image to the Tangram Pro container registry. You can run these containers to test your system.

We're also going to utilize end-to-end tests that are generated along side the interfaces and transform code.

5. LMCP Build Actions

First we'll configure Build Actions for the LMCP Component. We want to generate Interface Code for LMCP that we can run in a container. Specifically, we want to run the generated end-to-end test application named "Writer". In the Dockerfile provided below you'll see that we set the Transport to zeromq, move into test/end-to-end and run ./afrl_cmasi_AirVehicleStateWriter in its container.

1. Click Build in the top left menu
2. Select the RWS_LMCP Component
3. Go to the Actions tab and enable:
   • Generate Interface Code: C++
   • Compile Code: all
   • Containerize
   • Include Generated Code
   • Use Dockerfile Upload
4. Copy the code provided below into a plain text file named "Dockerfile"
5. Upload Dockerfile

Click to view Dockerfile code

docker
FROM tangramflex/pro-builder:latest AS build
ARG package=lmcp
ARG transport=zeromq
ARG transport_hostname=zeromq-transport
ARG binary=afrl_cmasi_AirVehicleStateWriter
ENV transport_hostname=${transport_hostname}
ENV package=${package}
ENV binary=${binary}
COPY code-gen/local_install /root/local_install
COPY code-gen/pkg_deps /root/pkg_deps
COPY code-gen/test /root/test
COPY code-gen/include /root/include
COPY code-gen/build /root/build
RUN sed -i "s/127.0.0.1/${transport_hostname}/g" /root/pkg_deps/genericapi/pkg_deps/transports-cpp/transports.config
RUN cd /root/test/end-to-end && make -j4 serializer=${package}
ENV LD_LIBRARY_PATH=/root/local_install/lib:/root/build/libs
WORKDIR /root/test/end-to-end
CMD [ "sh", "-c", "sleep 60 && ./${binary}" ]

6. STANAG Build Actions

Next, We want a Interface Code for STANAG4586 that we can run in a container. Specifically, we want to run the generated end-to-end test application named "Reader". In the Dockerfile provided below you'll see that we set the Transport to activemq, move into test/end-to-end and run ./Reader in its container.

1. Select the SS_STANAG Component
2. Go to the Actions tab and enable:
   • Generate Interface Code: C++
   • Compile Code: all
   • Containerize
   • Include Generated Code
   • Use Dockerfile Upload
3. Copy the code provided below into a plain text file named "Dockerfile"
4. Upload Dockerfile

Click to view Dockerfile code

docker
FROM tangramflex/pro-builder:latest AS build
ARG package=stanag4586
ARG transport=activemq
ARG transport_hostname=activemq-transport
ARG binary=Reader
ENV transport_hostname=${transport_hostname}
ENV package=${package}
ENV binary=${binary}
COPY code-gen/local_install /root/local_install
COPY code-gen/pkg_deps /root/pkg_deps
COPY code-gen/test /root/test
COPY code-gen/include /root/include
COPY code-gen/build /root/build
RUN sed -i "s/127.0.0.1/${transport_hostname}/g" /root/pkg_deps/genericapi/pkg_deps/transports-cpp/transports.config
RUN cd /root/test/end-to-end && make -j4 serializer=${package}
ENV LD_LIBRARY_PATH=/root/local_install/lib:/root/build/libs
WORKDIR /root/test/end-to-end
CMD [ "sh", "-c", "sleep 3 && ./${binary}" ]

7. Transform Build Actions

Lastly, we want a fully built Transform application that we can run in a container. In the Dockerfile provided below you'll see that we set the to and from properties accordingly, and run ./transform in its container.

1. Select the Transform
2. Go to the Actions tab
3. Choose the option Generate Transform with CSIs
4. Enable Containerize and remove the default file by clicking Remove Dockerfile
5. Copy the code provided below into a plain text file named "Dockerfile"
6. Upload Dockerfile

Click to view Dockerfile code

docker
FROM tangramflex/pro-builder:latest AS build
ARG from_package=lmcp
ARG to_package=stanag4586
ARG from_transport=zeromq
ARG to_transport=activemq
ARG from_transport_hostname=zeromq-transport
ARG to_transport_hostname=activemq-transport
ENV from_package=${from_package}
ENV to_package=${to_package}
ENV from_transport=${from_transport}
ENV from_transport_hostname=${from_transport_hostname}
ENV to_transport=${to_transport}
ENV to_transport_hostname=${to_transport_hostname}
# copy the generated transform code into the container
COPY out/transform /root/transform
COPY ./transform/${provider} /root/transform/
COPY out/${from_package} /root/${from_package}
COPY out/${to_package} /root/${to_package}
# disable contextual param error
RUN if [ 0 -lt $(ls /root/transform/Provider_*.hpp 2>/dev/null | wc -w) ]; then sed -i '/stderr/d' /root/transform/Provider_*.hpp; fi
RUN if [ 0 -lt $(ls /root/transform/Provider_*.hpp 2>/dev/null | wc -w) ]; then sed -i '/abort/d' /root/transform/Provider_*.hpp; fi
# rebuild app to apply changes
RUN make -C "/root/transform" clean
RUN make -C "/root/transform" in_transport=$from_transport out_transport=$to_transport
# hardcoded transport name
RUN sed -i "s/127.0.0.1/${from_transport_hostname}/g" /root/transform/transport_${from_transport}_in.config
RUN sed -i "s/127.0.0.1/${to_transport_hostname}/g" /root/transform/transport_${to_transport}_out.config
WORKDIR /root/transform
ENV LD_LIBRARY_PATH=/root/${from_package}/local_install/lib/:/root/${from_package}/build/libs/:/root/${to_package}/local_install/lib/:/root/${to_package}/build/libs/
CMD [ "sh", "-c", "sleep 2 && ./transform" ]

Run End-to-End Test

Great! Now you're all set and ready to generate code and test it out. In this section you'll:

• Complete a Build, which generates code and containers for the entire system
• Run an End-to-End Test in Containers, which will verify that the generated interfaces and transform are integrated

8. Generate Code

1. Click the blue Build button
2. Grab a cup of coffee ☕ and check out the streaming build logs. This may take up to 30 minutes to complete the first time as it is compiling the component interfaces, transform application and two transport libraries. Subsequent builds will use cached artifacts and be much quicker

tip

You can close Builds that are in progress at any time and return to the Design mode, or work on another project. A notification will pop up in Tangram Pro when the Build is complete. Click Builds on the top right menu to access all previous Builds.

9. Run Containers

Downloadable artifacts are available for completed Builds if you wish to run the code locally. Since we provided Dockerfiles, let's avoid that hassle and run the code inside Tangram Pro containers!

1. Select Run in the top left menu after the Build completes
2. Select the completed Build and click the Run button
3. Click the Run button

Tangram Pro will spin up the container images created during the build, and display each container's execution via system logs. You'll see 5 containers spin up and begin executing their commands. It may take a minute before they're all running.

How are the Containers Interacting?

1. ZeroMQ starts a proxy service
2. ActiveMQ starts a proxy service
3. LMCP Component executes the end-to-end test app "Writer" which sends a serialized AirVehicleState message to ZeroMQ and logs the message contents
4. Transform executes the transform app which listens to ZeroMQ for the AirVehicleState message, deserializes it, transforms it, serializes it in the STANAG format and sends it to ActiveMQ
5. STANAG Component executes the end-to-end test app "Reader" which listens for a serialized InertialStates message from ActiveMQ and logs the received message contents

If the STANAG container logs a message, then the integration was successful!

Cheers!

Your team has successfully completed the Integration Challenge using Tangram Pro 🎉 By completing this tutorial, you've learned how to integrate two systems that use different message standards, and messaging services (or Transports.)

Check out what you accomplished:

• Designed a Multi-Transport System
• Integrated different Message Types with a Transform
• Generated compiled Interfaces, Transform Code and Transport Libraries
• Created Containers and ran an End-to-End Transform Test inside them, all within Tangram Pro!

Additional User Guide Resources:

• Design
• Build
• Run
• Flex Transforms

If you want to know more about component-based system design in Tangram Pro, take a look at our blog posts.