Quantcast

Simulink Verification and Validation

Creating and Executing Test Cases

This demonstration shows how to use Simulink® Verification and Validation™ component verification functions to log input signals, create a harness model, and execute test cases.

The demonstration starts by logging input signals to the component implementing the controller in its parent model and creating harness model for the controller from that logged data. You author a new test case in the harness model. Then you capture all test cases and simulate the controller model for model coverage. Finally, you execute the controller with those test cases in simulation mode and Software-In-the-Loop (SIL) mode, and compare the results using CGV API.

Check Product Availability

This demonstration requires a valid Stateflow® license. To demonstrate test execution in Software-In-the-Loop (SIL) mode it also requires valid Simulink® Coder™ and Embedded Coder™ licenses.

if ~license('test','stateflow')
    return;
end

canUseSIL = license('test','real-time_workshop') && ...
    license('test','rtw_embedded_coder');

Logging Input Signals to the Component and Creating the Harness Model

The slvnvdemo_powerwindowslvnvdemo_powerwindow model contains a power window controller and a low-order plant model. The component slvnvdemo_powerwindow/power_window_control_system/controlslvnvdemo_powerwindow/power_window_control_system/control is a Model block that references the model slvnvdemo_powerwindow_controllerslvnvdemo_powerwindow_controller, which implements the controller with a Stateflow® chart.

To create a harness model for the controller with the signals that simulate the controller in the plant model, first log the input signals and then invoke harness generation with that logged data.

open_system('slvnvdemo_powerwindow');
load_system('slvnvdemo_powerwindow_controller');

loggedSignalsPlant = ...
    slvnvlogsignals('slvnvdemo_powerwindow/power_window_control_system/control');

harnessModelFilePath = ...
    slvnvmakeharness('slvnvdemo_powerwindow_controller',loggedSignalsPlant);
[~,harnessModel] = fileparts(harnessModelFilePath);

Author New Test Cases in Harness Model

You can modify the test data in a harness model by manually editing the data values using the Signal Builder user interface. You can also author new test cases by creating new signal groups in the block. Alternatively, you can use the signalbuildersignalbuilder command to accomplish the same thing programmatically.

load('slvnvdemo_powerwindow_controller_newtestcase.mat');

signalBuilderBlock = slvnvdemo_signalbuilder_block(harnessModel);
signalbuilder(signalBuilderBlock,'Append',...
    newTestTime, newTestData,...
    {'endstop','obstacle','driver(1)','driver(2)','driver(3)',...
    'passenger(1)','passenger(2)','passenger(3)'},'New Test Case');

Logging Test Cases of the Harness Model

In order to programmatically execute the model slvnvdemo_powerwindow_controllerslvnvdemo_powerwindow_controller with the test cases designed in the harness model, you first need to use the slvnvlogsignalsslvnvlogsignals function to capture the input values of all test cases in the necessary data format.

loggedSignalsHarness = slvnvlogsignals(harnessModel);
disp(loggedSignalsHarness);
    LoggedTestUnitInfo: [1x1 struct]
             TestCases: [1x2 struct]

Simulating Test Cases for Model Coverage

Use the slvnvruntestslvnvruntest function to simulate the controller with all of the test cases designed in the harness and measure model coverage.

runopts = slvnvruntestopts;
runopts.coverageEnabled = true;

[~, covdata] = slvnvruntest('slvnvdemo_powerwindow_controller',...
    loggedSignalsHarness,runopts);
cvhtml('Coverage with Test Cases from Harness', covdata);

Execute the Model in Simulation mode with CGV API

Use the slvnvruncgvtestslvnvruncgvtest function to execute the model slvnvdemo_powerwindow_controllerslvnvdemo_powerwindow_controller in simulation mode, with test cases captured from the harness model.

runopts = slvnvruntestopts('cgv');
disp(runopts);

runopts.cgvConn = 'sim';
cgvSim = slvnvruncgvtest('slvnvdemo_powerwindow_controller',...
    loggedSignalsHarness,runopts);
           testIdx: []
      outputFormat: 'TimeSeries'
    allowCopyModel: 0
       cgvCompType: 'topmodel'
           cgvConn: 'sim'

Starting execution:
  ComponentType: topmodel
  Connectivity: sim
  InputData:
  cgv_runtest\slvnvdemo_powerwindow_controller\slvnvdemo_powerwindow_controller_cgv_input_tc_1.mat
End CGV execution: status completed
Starting execution:
  ComponentType: topmodel
  Connectivity: sim
  InputData:
  cgv_runtest\slvnvdemo_powerwindow_controller\slvnvdemo_powerwindow_controller_cgv_input_tc_2.mat
End CGV execution: status completed

Execute the Model in Software-In-the-Loop (SIL) mode with CGV API

Now use the slvnvruncgvtestslvnvruncgvtest function to execute the model slvnvdemo_powerwindow_controllerslvnvdemo_powerwindow_controller in SIL mode, with test the same test cases.

if canUseSIL
    runopts.cgvConn = 'sil';
else
    % When SIL is not possible the demonstration runs another simulation.
    runopts.cgvConn = 'sim';
end
cgvSil = slvnvruncgvtest('slvnvdemo_powerwindow_controller',...
    loggedSignalsHarness,runopts);
Starting execution:
  ComponentType: topmodel
  Connectivity: sil
  InputData:
  cgv_runtest\slvnvdemo_powerwindow_controller\slvnvdemo_powerwindow_controller_cgv_input_tc_1_1.mat
### Starting build procedure for model: slvnvdemo_powerwindow_controller
### Successful completion of build procedure for model: slvnvdemo_powerwindow_controller
### Preparing to start SIL simulation ...
### Starting SIL simulation for component: slvnvdemo_powerwindow_controller
### Stopping SIL simulation for component: slvnvdemo_powerwindow_controller
End CGV execution: status completed
Starting execution:
  ComponentType: topmodel
  Connectivity: sil
  InputData:
  cgv_runtest\slvnvdemo_powerwindow_controller\slvnvdemo_powerwindow_controller_cgv_input_tc_2_1.mat
### Starting build procedure for model: slvnvdemo_powerwindow_controller
### Successful completion of build procedure for model: slvnvdemo_powerwindow_controller
### Preparing to start SIL simulation ...
### Starting SIL simulation for component: slvnvdemo_powerwindow_controller
### Stopping SIL simulation for component: slvnvdemo_powerwindow_controller
End CGV execution: status completed

Compare Results of Simulation and SIL Modes

The slvnvruncgvtestslvnvruncgvtest returns a cgv.CGV object after running tests. Use the CGV API to compare the results of executions in simulation and SIL modes for each test case designed in the harness model and show that they are equal.

for i=1:length(loggedSignalsHarness.TestCases)
    simout = cgvSim.getOutputData(i);
    silout = cgvSil.getOutputData(i);

    [matchNames, ~, mismatchNames, ~ ] = ...
        cgv.CGV.compare(simout, silout);

    fprintf('\nTest Case(%d):  %d Signals match, %d Signals mismatch', ...
        i, length(matchNames), length(mismatchNames));
end
Test Case(1):  2 Signals match, 0 Signals mismatch
Test Case(2):  2 Signals match, 0 Signals mismatch

Using CGV API to Detect Mismatches

Finally, you deliberately compare the results of executions for different test cases that shouldn't match and see how CGV API can be leveraged to detect and plot mismatches in results.

simout   = cgvSim.getOutputData(1);
silout   = cgvSil.getOutputData(2);
[matchNames , ~, mismatchNames, ~ ] = ...
        cgv.CGV.compare( simout, silout, 'Plot', 'mismatch');
fprintf( '\n%d Signals match, %d Signals mismatch', ...
    length(matchNames), length(mismatchNames));
disp( 'Mismatched Signal Names:');
disp(mismatchNames);

1 Signals match, 1 Signals mismatchMismatched Signal Names:
    'simDataTestCase.find('cgvlogsout').moveDown.Data'

Viewing and Comparing Model Outputs Graphically

Use the Simulation Data Inspector to plot model outputs and graphically compare the results of simulation and SIL execution. You launch the Simulation Data Inspector with the Simulation->Output->Simulation Data Inspector... menu item. You can load CGV results with the menu item File->Import data... on the Simulation Data Inspector interface. Select all eight entries related to these result variables:

simout1 = cgvSim.getOutputData(1);
silout1 = cgvSil.getOutputData(1);

simout2 = cgvSim.getOutputData(2);
silout2 = cgvSil.getOutputData(2);

Clean Up

To complete the demo, close all models.

close_system(harnessModel,0);
close_system('slvnvdemo_powerwindow');
close_system('slvnvdemo_powerwindow_controller');