sales@scenomics.com +1 650 396 9674

Implement Document Build Command

This exercise assumes you have completed the previous exercise. We provide some redundant instructions in order to help you know the correct location to add and modify code.

Modify Macro 'OctopusExecuteBuild'

  1. Go to the text editor and find the script app_octopus_scripts.ssl.
  2. Find the macro named OctopusExecuteBuild.

    macro OctopusExecuteBuild( CommandPresentationModuleInfo commandInfo )
    [Category="Build Commands", Guid="{B43182D4-3BA2-49FA-8404-31CE45BCA6D9}", Image=".\\icons\\generic_script_icon.bmp"]
    {
       if( LibAppServiceSettings.GetBoolParam( "ClearOutputWindowBeforeBuild" ) )
       {
          Console.Clear();
       }
    
       auto StrList a_slMessages;
       a_slMessages.AddBlank();
       LibAppOctopus.GenerateProjectBuildHeader(
          Model.Filename, a_slMessages );
    
       LibAppOctopus.Build( a_slMessages );
    
       a_slMessages.AddBlank();
       a_slMessages.Add( "Build completed." );
       LibAppOctopus.Out( a_slMessages );
    }
    

    We're going to modify this macro to perform the build. To do so, we'll replace a single line with new code.

  3. Replace the statement LibAppOctopus.Build( a_slMessages ); with the following code:

    Copy Text To Clipboard

    Render3D a_oAccel = Application.GetAccelerate3D();
    auto RenderInfo a_oRenderInfo = new RenderInfo( a_oAccel );
    
    // Get the GLSL compiler version.
    // Use the highest version supported by your GPU.
    int a_eMinGlslVersion = Enum.ShadingLanguageVersion_430();
    bool a_bCheckCompiler =  LibRender3D.CheckGlslCompiler(
       a_oAccel, a_eMinGlslVersion, a_slMessages );
    
    if( !( a_bCheckCompiler ) )
    {
       string a_sMessage =
          "GLSL compiler is NOT at least #version " + a_eMinGlslVersion;
       a_slMessages.Add( a_sMessage );
       a_slMessages.AddBlank();
       a_slMessages.Add( "Build failed!" );
    
       LibAppServiceBuild.Out( a_slMessages );
       return;
    }
    
    // Collect the WorkloadNodes from the document.
    auto NodeBuffer a_apJobs;
    auto NodeQuery a_oQuery;
    a_oQuery.QueryNode( Model, a_apJobs, WorkloadNode );
    
    // DEBUG
    //LibNodeBuffer.Out( a_apWorkloads );
    
    if( a_apJobs.GetCount() )
    {
       LibAppOctopus.Build( a_apJobs, commandInfo, a_slMessages );
    }
    else
    {
       string a_sMessage =
          "No <WorkloadNode> objects found in document. Nothing to build.";
       a_slMessages.Add( a_sMessage );
    }
    

    The finished macro looks like this:

    ///////////////////////////////////////////////////////////////////////////////
    // macro
    ///////////////////////////////////////////////////////////////////////////////
    
    function void OctopusExecuteBuild_OnUpdate( CommandPresentationModuleInfo commandInfo )
    {
       commandInfo.Status.SetHint( "Builds this document" );
    }
    
    macro OctopusExecuteBuild( CommandPresentationModuleInfo commandInfo )
    [Category="Build Commands", Guid="{A7FC822A-1B7B-42B5-BE06-450B7BE4819E}", Image=".\\icons\\generic_script_icon.bmp"]
    {
       if( LibAppServiceSettings.GetBoolParam( "ClearOutputWindowBeforeBuild" ) )
       {
          Console.Clear();
       }
    
       auto StrList a_slMessages;
       a_slMessages.AddBlank();
       LibAppOctopus.GenerateProjectBuildHeader(
          Model.Filename, a_slMessages );
    
       Render3D a_oAccel = Application.GetAccelerate3D();
       auto RenderInfo a_oRenderInfo = new RenderInfo( a_oAccel );
    
       // Do the build here...
       // Get the GLSL compiler version.
       // Use the highest version supported by your GPU.
       int a_eMinGlslVersion = Enum.ShadingLanguageVersion_430();
       bool a_bCheckCompiler =  LibRender3D.CheckGlslCompiler(
          a_oAccel, a_eMinGlslVersion, a_slMessages );
    
       if( !( a_bCheckCompiler ) )
       {
          string a_sMessage =
             "GLSL compiler is NOT at least #version " + a_eMinGlslVersion;
          a_slMessages.Add( a_sMessage );
          a_slMessages.AddBlank();
          a_slMessages.Add( "Build failed!" );
    
          LibAppServiceBuild.Out( a_slMessages );
          return;
       }
    
       // Collect the WorkloadNodes from the document.
       auto NodeBuffer a_apJobs;
       auto NodeQuery a_oQuery;
       a_oQuery.QueryNode( Model, a_apJobs, WorkloadNode );
    
       // DEBUG
       //LibNodeBuffer.Out( a_apWorkloads );
    
       if( a_apJobs.GetCount() )
       {
          LibAppOctopus.Build( a_apJobs, commandInfo, a_slMessages );
       }
       else
       {
          string a_sMessage =
             "No <WorkloadNode> objects found in document. Nothing to build.";
          a_slMessages.Add( a_sMessage );
       }
    
       a_slMessages.AddBlank();
       a_slMessages.Add( "Build completed." );
       LibAppOctopus.Out( a_slMessages );
    }

    First we check that the GLSL compiler is at least #version 430, which is the minimum version with compute shader support. If the version is less than 430, we print a build output message and return without doing any work. It's possible to have compute shader support for versions less than 430. If this is the case on your computer, then feel free to change the minimum version here to whatever will work for you.

    Then we use a <NodeQuery> object to populate a <NodeBuffer> object with pointers to all the <WorkloadNode> objects in the document. The <NodeBuffer> object is like a std::vector of pointers, but you can remove items from the collection without causing the collection to be resized. The stable sizing means that there won't be any memory allocations and/or copy operations when items are removed, which is better for performance in many situations. The <NodeQuery> object is a pretty simple as well: it allows you to search the hierarchy (using breadth first search) with a variety of filtering options, such as node type or other criteria.

    After collecting the <WorkloadNode> objects, the code invokes the build function for the collection. We pass to the following items to the build function: the <NodeBuffer> object representing the complete set of CPU or GPU compute workloads, the <CommandPresentationModuleInfo> object so we can update the status bar with messages during the build if desired, and a <StrList> object that stores log messages generated during the build.

    Separation-of-concerns is critical here, and throughout this implementation. We do not want the build system to know anything about the workloads it executes.

  4. Save changes to the script.

Implement Function 'CallBuildFunction'

  1. Go to the text editor and find the script app_octopus_util.ssl.
  2. Find the function named Out.

    ///////////////////////////////////////////////////////////////////////////////
    // function
    ///////////////////////////////////////////////////////////////////////////////
    
    function void Out( StrList p_slMessages )
    {
       for( int i = 0; i < p_slMessages.GetCount(); ++i )
       {
          Console.Out( p_slMessages.GetAt( i ) );
       }
    }
    

    We're going to implement a function that calls a script remotely. The remote script is what will actually perform the CPU and GPU compute workload. We must maintain a very clear separation-of-concerns here, so that these build system knows nothing about the workload it executes. That way we can scale this system out to perform a wide variety of CPU and GPU compute workloads.

  3. Insert the following code immediately below the function named Out.

    Copy Text To Clipboard

    ///////////////////////////////////////////////////////////////////////////////
    // function
    ///////////////////////////////////////////////////////////////////////////////
    
    function bool CallBuildFunction(
    
       ScriptFunction p_oTarget,
       WorkloadNode p_oJob,
       CommandPresentationModuleInfo p_oInfo,
       StrList p_slMessages
    
       )
    {
       for( int i = 0; i < p_oJob.ChildCount; ++i )
       {
          // Get the child node...
          Node a_oChild = p_oJob.Children[ i ];
    
          // Make sure it's derived from a FileNode.
          if( !( a_oChild.IsDerived( FileNode ) ) )
          {
             // Should we just skip?
             // We can skip for now.
             // Later we might need
             // to do something else.
             continue;
          }
    
          // Cast to a FileNode pointer.
          FileNode a_oDataSource = (FileNode)a_oChild;
          string a_sFileNodeName = a_oDataSource.Name;
          string a_sFileNodeType = "<FileNode>";
    
          // Get the source data path.
          auto FilePath a_oSrcDataPath =
             new FilePath( a_oDataSource.FilePath );
          a_oSrcDataPath.ResolveToModel( p_oJob );
    
          // Check if the source data exists.
          if( !( a_oSrcDataPath.FileExists() ) )
          {
             p_slMessages.Add(
                a_sFileNodeType + " source data does not exist: " +
                a_oSrcDataPath.GetPath() );
             continue;
          }
          else
          {
             p_slMessages.Add(
                a_sFileNodeType + " source data found: " +
                a_oSrcDataPath.GetPath() );
          }
    
          // Update the status bar...
          p_oInfo.Status.SetHint(
             "Executing workload on " + a_sFileNodeType + " '" +
             p_oJob.Name + "'" );
    
          // Configure the objects we're passing
          // as function parameters.
          auto VariantArray a_oParams;
          auto Variant a_oReturnValue;
          int a_nReturnVal = 0;
    
          int a_nParamCount = 3;
          a_oParams.Count = a_nParamCount;
          a_oParams.Objects[ 0 ].SetObject( a_oDataSource, FileNode );
          a_oParams.Objects[ 1 ].SetObject( p_oInfo, CommandPresentationModuleInfo );
          a_oParams.Objects[ 2 ].SetObject( p_slMessages, StrList );
    
          // Call the target function.
          bool a_bCallStatus = p_oTarget.Call( Script, a_oReturnValue, a_oParams );
    
          if( a_bCallStatus )
          {
             p_slMessages.Add(
                "Successfully executed " + a_sFileNodeType +
                " workload named '" + a_sFileNodeName + "'." );
          }
          else
          {
             p_slMessages.Add(
                "Failed to execute " + a_sFileNodeType +
                " workload named '" + a_sFileNodeName + "'." );
          }
       }
    
       return true;
    }

    This code starts out by running a loop on the <WorkflowNode> child nodes, which should all be of type <FileNode>. However, we must verify this, so we make sure to check the node type and we call continue if the node is not a <FileNode>.

    Then we cast the <Node> pointer to a <FileNode> pointer and verify that the source data exists. If it doesn't exist, we append the build log with an error message and call continue. We can't perform any work if the data doesn't exist.

    Finally we pack the function parameters into a <VariantArray> so that we can pass them to the remote function call. Using a remote function call here means that the function signature is the only detail that this code needs to know about the code that executes the CPU and GPU compute workloads. Furthermore, using a remote function call means the build system doesn't know that the workload execution code even exists. If the function doesn't exist

  4. Save changes to the script.

Modify Function 'Build'

  1. Go to the text editor and find the script app_octopus_util.ssl.
  2. Find the function named Build.

    ///////////////////////////////////////////////////////////////////////////////
    // function
    ///////////////////////////////////////////////////////////////////////////////
    
    function bool Build( StrList p_slMessages )
    {
       p_slMessages.Add( "Doing build work..." );
    
       return true;
    }
    

    We're going to implement a new function.

  3. Replace the entire function with the following:

    Copy Text To Clipboard

    ///////////////////////////////////////////////////////////////////////////////
    // function
    ///////////////////////////////////////////////////////////////////////////////
    
    function bool Build( NodeBuffer p_apJobs, CommandPresentationModuleInfo p_oInfo, StrList p_slMessages )
    {
       // Invoke the build command for each WorkloadNode.
       for( int i = 0; i < p_apJobs.GetCount(); ++i )
       {
          WorkloadNode a_oJob = (WorkloadNode)p_apJobs.Get( i );
    
          p_slMessages.AddBlank();
    
          // Skip invisible WorkloadNode objects.
          if( !( a_oJob.Visible ) )
          {
             p_slMessages.Add( " named '" + a_oJob.Name +
                "' is invisible and will be skipped." );
             continue;
          }
    
          // Return failure if there are no child nodes.
          if( !( a_oJob.ChildCount ) )
          {
             p_slMessages.Add( "<WorkloadNode> named '" + a_oJob.Name +
                "' contains no children! Unable to execute empty workload." );
             continue;
          }
    
          // Resolve the path to the script that performs the CPU or GPU compute
          // workload. This path comes in to this function as a relative path
          // such as '..\\..\\folder\\file.ssl' and we need an absolute path.
          auto FilePath a_oScriptPath = new FilePath( a_oJob.ScriptPath );
          a_oScriptPath.ResolveToModel( a_oJob );
    
          // Make sure we can actually load the script.
          // There might be SSL compiler errors.
          // This is going to compile the script code 'just in time'.
          // If the script fails to compile, the script engine
          // will print the errors in the output window, even
          // though you don't see code for it here.
          ScriptSource a_oSource = ScriptSource.LoadFromFile(
             a_oScriptPath.GetPath() );
          if( !( a_oSource ) )
          {
             string a_sMessage = "Unable to load script " +
                "specified by <WorkloadNode>: " + a_oScriptPath.GetPath();
             p_slMessages.Add( a_sMessage );
             delete a_oSource;
             continue;
          }
    
          // Find the actual build function.
          string a_sFunc = a_oJob.ScriptFunction;
          ScriptFunction a_oTarget = a_oSource.FindFunction( a_sFunc );
    
          if( !( a_oTarget ) )
          {
             string a_sMessage = "Unknown function: " +
                "specified by <WorkloadNode>: " + a_sFunc;
             p_slMessages.Add( a_sMessage );
             delete a_oSource;
             continue;
          }
    
          p_oInfo.Status.SetHint( "Building <WorkloadNode> '" + a_oJob.Name + "'" );
    
          CallBuildFunction( a_oTarget, a_oJob, p_oInfo, p_slMessages );
    
          p_slMessages.AddBlank();
          p_slMessages.Add( "Executed build for <WorkloadNode> named '" +
             a_oJob.Name + "'." );
    
          delete a_oSource;
       }
    
       p_oInfo.Status.SetHint( "Build complete" );
    
       return true;
    }

    This code starts out by resolving the relative path that the <WorkloadNode> specifies to the script that performs the CPU or GPU compute workload. (This is a Scenome Scripting Language script as you'll see in a moment.) We need an absolute path because we have to open the script and compile it.

    After creating an absolute path, we try to load the script from disk. The script file will be compiled during this process, and any script errors will be printed in the output window. Typically scripts are compiled when you select Desktop » Refresh Scripts from the main menu. In this case, because we must use a remote script call to maintain separation-of-concerns, we must also use just-in-time compilation. This can be a little difficult at first because you won't discover script errors until the build runs, but in practice it works quite well.

    Once the script is compiled, we search for the script function specified by the <WorkloadNode>, and print a build message if it is not found. In this case, although it is a serious error, we'll call continue so the loop keeps running while we try to see if we can successfully build anything.

    Last, we call CallBuildFunction( ... ), which makes the remote function call.

    It's a little wasteful to reload and recompile the script each time, but we don't have much choice here since each <WorkloadNode> can define its own workloads. It might be useful to sort workloads later, but we won't worry about that for now.

  4. Save changes to the script.

Modify function 'Execute'

  1. Go to the text editor and find the script app_octopus_workload_terrain_util.ssl.
  2. Find the function named Execute.

    ///////////////////////////////////////////////////////////////////////////////
    // function
    ///////////////////////////////////////////////////////////////////////////////
    
    function bool Execute( WorkloadNode p_oJob, CommandPresentationModuleInfo p_oInfo, StrList p_slMessages )
    {
       p_slMessages.Add( "Executed build..." );
    
       return true;
    }
    

    We're going to modify this function to perform the build.

  3. Replace the entire function with the following:

    Copy Text To Clipboard

    ///////////////////////////////////////////////////////////////////////////////
    // function
    ///////////////////////////////////////////////////////////////////////////////
    
    function bool Execute( FileNode p_oData, CommandPresentationModuleInfo p_oInfo, StrList p_slMessages )
    {
       p_slMessages.Add( "Processed elevation data for: " + p_oData.Name );
    
       return true;
    }
    

    This function is just a stub, but it allows us to check if the remote function call is successful.

  4. Save changes to the script.

Test Code Changes

  1. Return to the running Octopus app.
  2. Select Desktop » Refresh Scripts from the main menu. ( ALT + D + R )

    The application displays script compiler messages in the output window:

    Start loading scripts
    Done loading scripts; 10 loaded in 1.29 ms; avg 0.09
    

    If there are any script compiler errors, undo your changes in the text editor, go back to the previous step, and follow the instructions again. Here is an example of what error messages might look like:

    Start loading scripts
    D:\release6\scripts\app_shell_util.ssl(1770) : error: newline in constant
    Done loading scripts; 10 loaded in 1.29 ms; avg 0.09
    
  3. Select Graph » Build All from the main menu.

    The build runs and prints the build log to the output window.

    --- <Building Project 'D:\Release6\Content\Library\Octopus\Western-Washington-Mt-Baker-Terrain-Analysis\Western-Washington-Mt-Baker-Terrain-Analysis.box'> ---
    
    GLSL compiler version: 460
    <FileNode> source data found: D:\Release6\Content\Library\Textures\Elevation\USGS_13_n49w122.tif
    Processed elevation data for: USGS_13_n49w122.tif
    Successfully executed <FileNode> workload named 'USGS_13_n49w122.tif'.
    
    Executed build for <WorkloadNode> named 'Western-Washington-Mt-Baker-Terrain-Analysis'.
    
    Build completed.
    

    The build command now works, but of course it still doesn't do any useful work. Next, we'll do the complete implementation of this function so that it executes the CPU and GPU compute workload.

    This exercise is complete. Please proceed to the next exercise.