Skip to main content

How-To Guides

This guide provides instructions for performing common tasks in Tangram Pro Verifier. Please use the sidebar to navigate to the topics you're looking for.

Prepare Your Component for Verifier​

See What You'll Need in the Verifier reference guide.

To use Verifier, you'll need to create or edit a component in a Tangram Pro project while in Design mode.

  1. Configure the messages that your component will need to send and receive
    • To use a new set of messages, you'll need to create a new Flex package
  2. Set up the transport that messages will travel over

Create Message Sequences​

  1. Open a project and go to Verify
  2. Select a component and click the Component Sequences tab
  3. Click Add Sequence
  4. Click Add Initial Sequence Message and choose the first message that you want to either be input to your component-under-test or output by it

Add Messages to a Sequence​

  1. Click the + Add Initial Message on an empty sequence, or hover over the dot menu to the left of any existing message in a sequence
  2. Choose Add Message Before or Add Message After

Add Message

Create an Alternate Message Path​

  1. Click on the dot to the left of the message that you want to define an alternate for
  2. Choose Add Alternate Message
  3. You can then add messages to each separate path as needed

Alt Message

Duplicate Sequences​

You can create a copy of a sequence, including its constraints, by clicking the Duplicate icon on the top right of a sequence.

Create and Use Message Templates​

  1. Click the Message Templates button in the top right of the Component Sequences panel, next to the + Add Sequence button
  2. Create a new template, then select the package and message or struct which will have its fields templated
  3. Type in the constraints YAML you'd like to template and save

This template can now be applied directly to any message using the menu located at the bottom left of the Constraints panel, or using the anyOf constraint.

Run Verifier Tests​

When you start a test, Tangram Pro will iterate through all message sequences the component should be able to handle, analyzing whether or not the component behaves as expected. Tangram Pro will listen when the component should be sending a message, and will generate a message per the sequence criteria when the component expects to receive a message from the system.

Setup A New Test​

Each time you start a test you have the option of changing the setup. Your previous setup is remembered so you don't have to set it everytime.

Setup New Test

To start a test​

  1. Click the Setup New Test button
  2. Specify the container image that contains your component software
    • Choose Use Project Build to use an image generated by a Tangram Pro build
    • Choose Use Container Registry to use an image in a registry
  3. Enter Test Runs Total: The number of times you want to test your sequences (e.g. 2 means your sequences will be tested twice)
  4. Enter Max Path Retries: The number of retries to attempt if a sequence is unsatisfied (e.g. 5 means a sequence path will be attempted up to 5 times until it is satisfied. If it is satisified on the 2nd attempt it will not retry again.)
  5. Enter Message Timeout: The time in milliseconds to wait for the next expected message in the sequence
  6. Choose sequences to Skip by disabling them
  7. Click Begin Verifier Test

Tangram Pro will run the component in isolation, with the test environment emulating the rest of the system, to test the component's interface and see how the it reacts to its environment.

Rerun Test​

You can rerun any completed test with the click of a button. This will execute the test again with the same setup and input parameters.

  1. Go to View Tests
  2. Open a completed test
  3. Click the Run Test Again button in the top right

Add Field Constraints to a Message​

  1. Locate a message in a sequence and click the Constraints link beside it
  2. Type in constraints YAML, apply a message template, or both!

If you do both, apply a message template and type in constraints, the typed-in constraints will override any constraints from the template if the field names match.

Constraints

View Tests​

When a test is running, Tangram Pro provides live reporting as it processes and analyzes each sequence, including the contents of the messages being sent. When complete, the test summary is saved for you to view all of those details later.

Interpreting Results​

Tangram Pro reports a sequence as "Satisfied" if both of these conditions are met:

  • The messages were sent and received in the order specified by sequences
  • The message contents met all specified constraints

A sequence will be "Unsatisfied" if either of the above conditions are not met. It is important to note, a sequence may be "Unsatisfied" if the wait time between subsequent messages exceeds the "Message Timeout" and if the number of retries reaches the "Max Path Retries".

Tangram Pro also provides two real-time log streams.

  • Component Logs: These are the logs output by your software component
  • Debug Logs: These are the logs output by Tangram Pro's test environment

In the event that a test results in "Error", the logs may be helpful in determining the cause.

Viewing Prior Tests​

  1. Go to View Tests
  2. Click on a test to view its results in more detail

Export Test Results​

Completed test results can be exported in either PDF, DOCX, or JSON format.

  1. Go to View Tests and open a completed test
  2. Click the Export Results button on the top right
  3. Wait for the process to complete (this may take a few seconds)
  4. Then click the Download Results button and choose the desired format