sales@scenomics.com +1 650 396 9674


Release details are described below:

Iterator Changes

Completed Phase 2 of the conversion to iterator-based algorithms. This included targeted refactoring of the Scenome Scripting Language codebase and testing of all affected systems.

The first significant change is the addition of array view classes that wrap a pair of iterators. This provides necessary syntax sugar to significantly reduce the amount of code required to use iterators.

You can work with the <ArrayViewT> types if you want shorter code. This type wraps a pair of <TypedIterator> objects of the related type. There are data interfaces for array objects ( SSL::GetView() ) that return an <ArrayViewT> object that wraps a pair of iterators. For example auto Float32ArrayView src_view = src_data.GetView(); wraps declaration and initialization/binding into a single line of code. This syntax almost matches the terseness of the previous algorithm system.


   // The number of interpolated values to generate.
   int a_nOutputCount = 5;

   // The values to linear interpolate.
   auto Float32Array src_data;
   src_data.Add( 0 );
   src_data.Add( 1.0 );

   auto Float32ArrayView src_view = src_data.GetView();

   auto Float32Array dst_data = new Float32Array( a_nOutputCount );
   auto Float32ArrayView dst_view = dst_data.GetView();

   auto Float32ArrayAlgorithms a_afAlgorithms;
   a_afAlgorithms.LinearInterpolate(
      src_view.First, src_view.Last, dst_view.First, dst_data.GetCount() );

   LibFloat32Array.Out( dst_data );

   /* Output will be...
   Executing: LinearInterpolate
   FLOAT32ARRAY OUTPUT generated by TYPE_FLOAT32_ARRAY_UTIL.SSL
   0,
   0.25,
   0.5,
   0.75,
   1
   */
                           

This example finds the min and max values in a 32-bit floating-point <Image>.


   // Open a file from disk into an <Image> object.
   auto FilePath a_oImagePath = new FilePath(
      LibAppServiceMain.Get2DTexturesPath() );
   a_oImagePath.AppendPath( "IPF_FP32x4.image" );

   auto Image src;
   src.OpenFile( a_oImagePath.GetPath() );

   // Take a view of the object.
   auto Float32ArrayView src_view = src.GetFloat32View();

   // Allocate storage for min/max values.
   auto Float32Array dst = new Float32Array( 2, 0.0 );
   auto Float32ArrayView dst_view = dst.GetView();

   // Find the min/max values in the <Image> object's underlying array.
   auto Float32ArrayAlgorithms a_afAlgorithms;
   a_afAlgorithms.FindMinMax( src_view.First, src_view.Last, dst_view.First );

   LibFloat32Array.Out( dst );

   /* Output will be...
   Executing: FindMinMax
   FLOAT32ARRAY OUTPUT generated by TYPE_FLOAT32_ARRAY_UTIL.SSL
   0.36666,
   1.0
   */
                           
Parallel ForEach

The second significant change is the deprecation of many previous array algorithms implementations and the replacement of these with a new algorithm SSL::ParallelForEach that allows you to write GLSL lambdas that run on collections. In the vast majority of common use cases, this offers great performance improvements over the C++ equivalent. Our benchmark tests involve 64MB <Image> objects. There is a cost to send and receive the data from the GPU and it is possible that in some cases, these costs can exceed the savings offered by using compute shaders to mutate the data.

As with their C++ counterparts, it's possible to invalidate iterators. If you increase the underlying allocation, your iterator range will be short compared to the underlying collection, but, technically speaking, the iterators may remain stable. If you decrease the size of the underlying collection, your iterators may be out of range. Either way, any attempt to use/dereference an invalid iterator results in the iterator being cleared to constructor defaults.


   // Create source data.
   auto Float32Array dst;
   dst.Count = p_nDstCount;

   auto Float32ArrayView dst_view = dst.GetView();

   dst.Count = 2; // Invalidates the iterator if p_nDstCount is not also equal to 2.

   int a_nPerChannel = p_nDstCount / p_nChannelCount;

   auto Float32ArrayAlgorithms a_afAlgorithms;

   int a_nBase = 0;
   for( int i = 0; i < p_nChannelCount; ++i )
   {
      auto Float32Array src = new Float32Array( a_nPerChannel, 0.0 );
      auto Float32ArrayView src_view = src.GetView();

      a_afAlgorithms.Iota( src_view.First, src_view.Last, a_nBase );
      a_nBase += p_nChannelCount;

      // This fails. Invalid iterator cleared to constructor defaults on dereferencing.
      a_afAlgorithms.Interleave(
         src_view.First, src_view.Last, p_nChannelCount, i, dst_view.First );

      LibFloat32Array.Out( src );
   }

   LibFloat32Array.Out( dst );
                           

The existing algorithm system provides basic support algorithms that are required for the application to operate without the GPU. As we implemented additional algorithms, it became clear that we couldn't easily implement the full range of algorithms that people need. In response to this, we implemented a generic algorithm, SSL::ParallelForEach that allows you to write compute shader lambdas in GLSL that operate over collections of primitive types.


   Texture a_oTexture = LibSelect.FirstTexture();
   Image a_oImage = a_oTexture.GetFirstImage();

   // Take a view of the object.
   auto Float32ArrayView src_view = a_oImage.GetFloat32View();

   // Set the first and last positions.
   //src_view.First.Position = 100000;
   //src_view.Last.Position = 32;

   int a_nStartTickCount = LibAppServicePerformance.Start();

   // Get the rendering device.
   Render3D a_oAccel = Application.GetAccelerate3D();

   // Configure the lambda parameters. This tells
   // the ParallelForEach algorithm how to build
   // the compute shader.
   auto Float32ArrayAlgorithms a_afAlgorithms;
   auto ComputeLambdaParams a_oLambda;
   a_oLambda.Device = a_oAccel;
   a_oLambda.Domain = 1;
   a_oLambda.DispatchLocalGroups = true;
   a_oLambda.WorkGroupSizeX = 4096;
   a_oLambda.WorkGroupSizeY = 1;
   a_oLambda.WorkGroupSizeZ = 1;
   a_oLambda.LocalGroupSizeX = 1024;
   a_oLambda.LocalGroupSizeY = 1;
   a_oLambda.LocalGroupSizeZ = 1;
   a_oLambda.ArrayDimensionX = 0;
   a_oLambda.ArrayDimensionY = 0;
   a_oLambda.ArrayDimensionZ = 0;

   // Fill an array of float32 values.
   //a_oLambda.VectorDimension = 1;
   //a_oLambda.Source = "data[pos] = float( 1.0 )";

   // Fill an array of float32 values bound as an array of GLSL vec4.
   //a_oLambda.VectorDimension = 4;
   //a_oLambda.Source = "data[pos] = normalize( data[pos] )";

   // Implement a complete algorithm that transforms
   // the values in the source array to a new range.
   a_oLambda.VectorDimension = 4;
   string a_sLambda =
      "   float newMin = float( 0.0 );\n" +
      "   float newMax = float( 1.0 );\n" +
      "   float currMin = float( " + a_oTexture.ImageDataMin + " );\n" +
      "   float currMax = float( " + a_oTexture.ImageDataMax + " );\n" +
      "   data[pos] = ( ( newMax - newMin ) * ( data[pos] - currMin ) ) / ( currMax - currMin ) + newMin";
   a_oLambda.Source = a_sLambda;

   // Execute the lambda.
   a_afAlgorithms.ParallelForEach(
      src_view.First, src_view.Last, a_oLambda );

   // Print the source code we generated.
   Console.Out( a_oLambda.GeneratedSource );

   PrintPerf( a_nStartTickCount, a_sFunc );

   // Print N values...
   //src_view.First.Position = 0;
   //src_view.Last.Position = 4;
   //LibFloat32ArrayAlgorithms.Out( src_view.First, src_view.Last );
                              

The algorithm builds a compute shader, executes the shader, and returns. You can print the compute shader code that was used to modify the collection.


   #extension GL_ARB_compute_variable_group_size : enable
   layout( local_size_variable ) in;

   buffer data_buffer
   {
      vec4 data[4194304];
   };

   void main(void)
   {
      int pos = int( gl_GlobalInvocationID.x );
      float newMin = float( 0.0 );
      float newMax = float( 1.0 );
      float currMin = float( 0.145098045468 );
      float currMax = float( 0.952941179276 );
      data[pos] = ( ( newMax - newMin ) * ( data[pos] - currMin ) ) / ( currMax - currMin ) + newMin;
   }
                           
View System Types

Array view classes are strongly-typed and not available for polymorphic access through a base class. Supported view types are as follows:

View System Type Base Class
<Int8ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Int16ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Int32ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Int64ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Uint8ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Uint16ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Uint32ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Uint64ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Float16ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Float32ArrayView> <ArrayViewT> Base class not accessible in SSL.
<Float64ArrayView> <ArrayViewT> Base class not accessible in SSL.
View Binding Support

SSL types with view binding support are as follows:

View Binding Support Status
<Image> Supported
<Int8Array> Supported
<Int16Array> Supported
<Int32Array> Supported
<Int64Array> Supported
<Uint8Array> Supported
<Uint16Array> Supported
<Uint32Array> Supported
<Uint64Array> Supported
<Float16Array> Supported
<Float32Array> Supported
<Float64Array> Supported
<Float32Matrix> Supported
<Float64Matrix> Supported
<Float32MatrixArray> Not Yet Supported. Probably Scenome 21.7 or Scenome 21.8.
<Float64MatrixArray> Not Yet Supported. Probably Scenome 21.7 or Scenome 21.8.