 Code Coverage - Maple Help

Example: A Package for Code Coverage Profiling Introduction

 The following example is a package called coverage. The coverage package defines procedures and modules for coverage profiling, that is, it turns on statement-level tracing. It also serves as an example of a small package and illustrates ways in which modules can be manipulated.
 Note: The Maple CodeTools package provides tools for profiling code and testing code coverage. For more information, refer to CodeTools. Design

 You can write tests that exercise each part of a program to ensure that the program:
 • Works correctly
 • Continues to work when it, or other programs on which it depends, changes over time.
 It is therefore important to determine whether or not each statement in a procedure has been covered by a test case. The traceproc option of the Maple command debugopts provides this capability. It takes the name p of a procedure, using the syntax

 debugopts( traceproc = p );

 and traces this procedure for coverage profiling. Here is an example.
 > p := proc( x )     if x < 0 then         2 * x;     else         1 + 2 * x;     end if; end proc:
 > debugopts( 'traceproc' = p ):
 Once the procedure has been marked for coverage profiling, profiling information at the statement level is stored each time the procedure is executed. To view the profiling information, use the procedure showstat.
 > p( 2 );
 ${5}$ (1)
 > showstat( p );
 p := proc(x)      |Calls Seconds  Words| PROC |    1   0.000      3|    1 |    1   0.000      3| if x < 0 then    2 |    0   0.000      0|     2*x                             else    3 |    1   0.000      0|     1+2*x                             end if end proc
 The display shows that only one branch of the if statement that forms the body of p was taken so far. This is because only a non-negative argument has been supplied as an argument to p. To get complete coverage, a negative argument must also be supplied.
 > p( -1 );
 ${-2}$ (2)
 > showstat( p );
 p := proc(x)      |Calls Seconds  Words| PROC |    2   0.000      6|    1 |    2   0.000      6| if x < 0 then    2 |    1   0.000      0|     2*x                             else    3 |    1   0.000      0|     1+2*x                             end if end proc
 The display shows that each statement in the body of p has been reached.
 To display the profiling information, use the debugopts command with the traceproctable=procedure_name equation argument.
 > debugopts( traceproctable=p );
 $\left[\begin{array}{ccc}{2}& {0}& {6}\\ {2}& {0}& {6}\\ {1}& {0}& {0}\\ {1}& {0}& {0}\end{array}\right]$ (3)
 The coverage package illustrated in this example extends this functionality to modules and acts as an interface to the debugopts with the traceproc option.
 The coverage package has two exports: profile and covered. Two private procedures, rprofile and traced, are used as subroutines. They are stored in local variables of the underlying module of the package. The Package Source

 Here is the source code for the package.
 > coverage := module()     description "a package of utilities for "                 "code coverage profiling";     option package;     export profile, covered;     local rprofile, traced, userprocs;     # Instrument a procedure or module     # for coverage profiling. Return the     # number of procedures instrumented.     profile := proc()         local arg;         add( rprofile( arg ), arg = [ args ] );     end proc;     rprofile := proc( s::name )         local e;         if type( s, 'procedure' ) then             debugopts( 'traceproc' = s );             1;         elif type( s, 'module' ) then             add( procname( e ),                e = select( type,                      [ exports( s, 'instance' ) ],                      '{ module, procedure }' ) );         else             error "only procedures and modules can be profiled";         end if;     end proc;     # Subroutine to recognize non-builtin procedures     userprocs := proc( s)         type( 's', procedure) and not( type( 's', builtin ) );     end proc;     # Subroutine to recognize profiled procedures     traced := proc( s )         debugopts( 'istraceproced' = 's' );     end proc;     # Determine which procedures have     # coverage information.     covered := proc()         local S;         S := [ anames( ) ];         S := select( userprocs, S );         S := select( traced, S );         if nargs > 0 and args[ 1 ] = 'nonzero' then             S := select( s -> evalb( s[1,1] <> 0 ), S );         elif nargs > 0 then             error "optional argument is the name nonzero";         end if;         map( parse, map( convert, S, 'string' ) );     end proc; end module: How the Package Works

 The export profile is an interface to the package's principal facility: implementing procedures and modules for code coverage profiling. It returns the number of procedures being traced and calls the private subroutine rprofile to do most of the work.
 1 The procedure rprofile accepts a name s as an argument. If s is the name of a procedure, rprofile simply calls debugopts to trace the procedure assigned to that name. Otherwise, if s is the name of  a module, rprofile selects any exports of the module that are procedures or modules and calls itself recursively to trace them. If the parameter s is assigned a value of any other type, an exception is raised.
 2 The expression [ exports( s, 'instance' ) ] evaluates to a list of all the exported variables of the module that are assigned to s. It is important to pass the instance option to exports, because when those names are passed to rprofile in a recursive call, rprofile must test the type of their assigned values. This list contains all the module exports, so those that are of type procedure, or of type module, are selected by using a call to select. The recursion is effected in the call to add, which sums the return values of all the recursive calls to rprofile.
 3 The exported procedure covered is used to determine which procedures have been traced and called, with profiling information stored. One possible design would store this information in a private table in the coverage package. With this design, covered could simply query that internal table for the names of the procedures that have been traced and that have profiling information stored. However, a user may have traced the procedure manually by calling debugopts directly, or historical profiling data may have been read from a Maple repository. Therefore, a design that queries the system directly, without regard to how a procedure was initially traced, is best used.
 The procedure covered queries Maple for all the names currently assigned values using the Maple command anames ("assigned names"). Names corresponding to profiled user procedures are selected using the subroutines userprocs and traced. If the nonzero option is passed to covered, then only those which have actually been called are chosen. The final statement
 > map( parse, map( convert, S, 'string' ) );
 first converts the names to strings, and then calls parse on each string to convert it to the procedure for which profiling data is stored. Using the Package

 As with all packages, you can access the coverage package interactively by using the with command.
 > with( coverage );
 $\left[{\mathrm{covered}}{,}{\mathrm{profile}}\right]$ (4)
 A list of the package exports is returned. Alternatively, the package exports can always be accessed by using the long forms coverage:-profile and coverage:-covered.
 Suppose that you want to test the procedure copy (chosen because it is short). The copy procedure produces a new copy of a table, array, or rtable. Now that the coverage package has been globally imposed by using with, simply call
 > profile( copy );
 ${1}$ (5)
 The return value of 1 indicates that, as expected, one procedure is being traced. Next, call copy with a few arguments (output suppressed):
 > copy( table() ):
 > copy( array( 1 .. 3 ) ):
 Using covered, copy has its profiling information stored.
 > covered( 'nonzero' );
 $\left[{p}{,}{\mathrm{copy}}\right]$ (6)
 From the output of showstat,
 > showstat( copy );
 copy := proc(A) local B, G, H, X, str;      |Calls Seconds  Words| PROC |    2   0.000    536|    1 |    2   0.000      6| if 1 < _npassed and _passed = (':-deep') then    2 |    0   0.000      0|     if type(A,'{module, table, procedure, moduledefinition}') then    3 |    0   0.000      0|         op(1,sscanf(sprintf("%m",eval(A)),"%m"))                                 else    4 |    0   0.000      0|         B := op(1,sscanf(sprintf("%m",A),"%m"));    5 |    0   0.000      0|         if B::(':-rtable') and B::(':-attributed') then    6 |    0   0.000      0|             G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));    7 |    0   0.000      0|             if 0 < numelems(G) then    8 |    0   0.000      0|                 setattribute(B,seq(H))                                         end if                                     end if;    9 |    0   0.000      0|         return B                                 end if                             elif type(A,'rtable') then   10 |    0   0.000      0|     B := rtable(rtable_indfns(A),rtable_dims(A),A,rtable_options(A),(':-readonly') = (':-false'));   11 |    0   0.000      0|     if B::(':-attributed') then   12 |    0   0.000      0|         G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));   13 |    0   0.000      0|         if 0 < numelems(G) then   14 |    0   0.000      0|             setattribute(B,seq(H))                                     end if                                 end if;   15 |    0   0.000      0|     return B                             elif type(A,'table') then   16 |    2   0.000      0|     if type(A,'array') then   17 |    1   0.000    265|         array(A)                                 elif type(A,'cache') then   18 |    0   0.000      0|         Cache(A)                                 else   19 |    1   0.000    265|         table(A)                                 end if                             elif type(A,'procedure') then   20 |    0   0.000      0|     subs(X = X,eval(A))                             elif type(A,'{module, moduledefinition}') then   21 |    0   0.000      0|     if type(A,'record') then   22 |    0   0.000      0|         Record(eval(A))                                 elif type(A,'object') then   23 |    0   0.000      0|         Object(A)                                 else   24 |    0   0.000      0|         error "cannot copy a module or module definition"                                 end if                             else   25 |    0   0.000      0|     A                             end if end proc
 it appears that the rtable case (statement 2) has not been called. Add a test for the rtable case.
 > copy( rtable() ):
 > showstat( copy );
 copy := proc(A) local B, G, H, X, str;      |Calls Seconds  Words| PROC |    3   0.000    634|    1 |    3   0.000      9| if 1 < _npassed and _passed = (':-deep') then    2 |    0   0.000      0|     if type(A,'{module, table, procedure, moduledefinition}') then    3 |    0   0.000      0|         op(1,sscanf(sprintf("%m",eval(A)),"%m"))                                 else    4 |    0   0.000      0|         B := op(1,sscanf(sprintf("%m",A),"%m"));    5 |    0   0.000      0|         if B::(':-rtable') and B::(':-attributed') then    6 |    0   0.000      0|             G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));    7 |    0   0.000      0|             if 0 < numelems(G) then    8 |    0   0.000      0|                 setattribute(B,seq(H))                                         end if                                     end if;    9 |    0   0.000      0|         return B                                 end if                             elif type(A,'rtable') then   10 |    1   0.000     71|     B := rtable(rtable_indfns(A),rtable_dims(A),A,rtable_options(A),(':-readonly') = (':-false'));   11 |    1   0.000     24|     if B::(':-attributed') then   12 |    0   0.000      0|         G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));   13 |    0   0.000      0|         if 0 < numelems(G) then   14 |    0   0.000      0|             setattribute(B,seq(H))                                     end if                                 end if;   15 |    1   0.000      0|     return B                             elif type(A,'table') then   16 |    2   0.000      0|     if type(A,'array') then   17 |    1   0.000    265|         array(A)                                 elif type(A,'cache') then   18 |    0   0.000      0|         Cache(A)                                 else   19 |    1   0.000    265|         table(A)                                 end if                             elif type(A,'procedure') then   20 |    0   0.000      0|     subs(X = X,eval(A))                             elif type(A,'{module, moduledefinition}') then   21 |    0   0.000      0|     if type(A,'record') then   22 |    0   0.000      0|         Record(eval(A))                                 elif type(A,'object') then   23 |    0   0.000      0|         Object(A)                                 else   24 |    0   0.000      0|         error "cannot copy a module or module definition"                                 end if                             else   25 |    0   0.000      0|     A                             end if end proc
 Statement 4 has not been called. This statement can be reached by assigning an array or table to a name and by calling copy with that name as argument.
 > t := table():
 > copy( t ):
 > showstat( copy );
 copy := proc(A) local B, G, H, X, str;      |Calls Seconds  Words| PROC |    4   0.000    906|    1 |    4   0.000     12| if 1 < _npassed and _passed = (':-deep') then    2 |    0   0.000      0|     if type(A,'{module, table, procedure, moduledefinition}') then    3 |    0   0.000      0|         op(1,sscanf(sprintf("%m",eval(A)),"%m"))                                 else    4 |    0   0.000      0|         B := op(1,sscanf(sprintf("%m",A),"%m"));    5 |    0   0.000      0|         if B::(':-rtable') and B::(':-attributed') then    6 |    0   0.000      0|             G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));    7 |    0   0.000      0|             if 0 < numelems(G) then    8 |    0   0.000      0|                 setattribute(B,seq(H))                                         end if                                     end if;    9 |    0   0.000      0|         return B                                 end if                             elif type(A,'rtable') then   10 |    1   0.000     71|     B := rtable(rtable_indfns(A),rtable_dims(A),A,rtable_options(A),(':-readonly') = (':-false'));   11 |    1   0.000     24|     if B::(':-attributed') then   12 |    0   0.000      0|         G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));   13 |    0   0.000      0|         if 0 < numelems(G) then   14 |    0   0.000      0|             setattribute(B,seq(H))                                     end if                                 end if;   15 |    1   0.000      0|     return B                             elif type(A,'table') then   16 |    3   0.000      0|     if type(A,'array') then   17 |    1   0.000    265|         array(A)                                 elif type(A,'cache') then   18 |    0   0.000      0|         Cache(A)                                 else   19 |    2   0.000    534|         table(A)                                 end if                             elif type(A,'procedure') then   20 |    0   0.000      0|     subs(X = X,eval(A))                             elif type(A,'{module, moduledefinition}') then   21 |    0   0.000      0|     if type(A,'record') then   22 |    0   0.000      0|         Record(eval(A))                                 elif type(A,'object') then   23 |    0   0.000      0|         Object(A)                                 else   24 |    0   0.000      0|         error "cannot copy a module or module definition"                                 end if                             else   25 |    0   0.000      0|     A                             end if end proc
 The only case that has not been called is the one in which the argument to copy is something other than an rtable, array, or table.
 > copy( 2 ):
 > showstat( copy );
 copy := proc(A) local B, G, H, X, str;      |Calls Seconds  Words| PROC |    5   0.000    909|    1 |    5   0.000     15| if 1 < _npassed and _passed = (':-deep') then    2 |    0   0.000      0|     if type(A,'{module, table, procedure, moduledefinition}') then    3 |    0   0.000      0|         op(1,sscanf(sprintf("%m",eval(A)),"%m"))                                 else    4 |    0   0.000      0|         B := op(1,sscanf(sprintf("%m",A),"%m"));    5 |    0   0.000      0|         if B::(':-rtable') and B::(':-attributed') then    6 |    0   0.000      0|             G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));    7 |    0   0.000      0|             if 0 < numelems(G) then    8 |    0   0.000      0|                 setattribute(B,seq(H))                                         end if                                     end if;    9 |    0   0.000      0|         return B                                 end if                             elif type(A,'rtable') then   10 |    1   0.000     71|     B := rtable(rtable_indfns(A),rtable_dims(A),A,rtable_options(A),(':-readonly') = (':-false'));   11 |    1   0.000     24|     if B::(':-attributed') then   12 |    0   0.000      0|         G, H := selectremove(type,[attributes(B)],(':-identical')(':-source_rtable') = (':-rtable'));   13 |    0   0.000      0|         if 0 < numelems(G) then   14 |    0   0.000      0|             setattribute(B,seq(H))                                     end if                                 end if;   15 |    1   0.000      0|     return B                             elif type(A,'table') then   16 |    3   0.000      0|     if type(A,'array') then   17 |    1   0.000    265|         array(A)                                 elif type(A,'cache') then   18 |    0   0.000      0|         Cache(A)                                 else   19 |    2   0.000    534|         table(A)                                 end if                             elif type(A,'procedure') then   20 |    0   0.000      0|     subs(X = X,eval(A))                             elif type(A,'{module, moduledefinition}') then   21 |    0   0.000      0|     if type(A,'record') then   22 |    0   0.000      0|         Record(eval(A))                                 elif type(A,'object') then   23 |    0   0.000      0|         Object(A)                                 else   24 |    0   0.000      0|         error "cannot copy a module or module definition"                                 end if                             else   25 |    1   0.000      0|     A                             end if end proc
 The final output shows that every statement has been reached by the test cases. This functionality is very useful for interactively developing unit tests for Maple programs.
 Note: The source presented here for the coverage package has been simplified for presentation in printed form. The full source code is available in the samples/ProgrammingGuide directory of the Maple installation. See Also