Scripting

Scripts

You can create queries and extend part of the application using the domain specific language called IHDSL. To write a new query, open a script terminal window, define the data source you will use (you can define new ones), and write your script.
To do this, the context menu has support for IHDSL commands, command templates, available field names and fields, and the power of the Smalltalk dynamic environment to inspect and debug objects in runtime.

The editor allows double-click to select statements in parentheses, evaluate them and show the result, for example to be able to explore and have feedback on the intermediate results of a script containing nested queries.

You can save the script to a file on the disk (then you can access it from the context menu of the desktop) or add it to the bundles (see next section) and access it directly from the main bar menu). You can comment the function of the script in the first lines of the script by putting the text in parentheses. If the script has a comment, it will be displayed as a ballon text when selected in that menu.

If the script generates a canonical or default result, that is, it returns a ResultSet with the default fields of an indicator (project, period, date, value), can add it to the indicator repository and can be used as any other indicator the application. You can also create IHDSL-based bundles from the Model menu using the Bookmarks Definition Browser

Fig. 44. Adding a new indicator to the repository based on a script

Open

Script bundles

The scripts library, or bundles, are IHDSL scripts stored in the bundles folder of the application and can be accessed directly from the bundles menu of the main menu bar.

Simply select the menu scripts to execute useful queries or statements immediately. Note that the script may or may not display a result (ResultSet/s or graph/s), in fact they are Smalltalk / IHDSL statements.

You can edit the script by simultaneously pressing the SHIFT key. Comment your scripts to view the comment when selecting each item.

The menu has two sections separated by a line. The first is the scripts you have added to the library. The second contains smart category submenues and IHDSL filters that you can also execute or edit by directly accessing them

Fig. 45. Scripts bundles access menu

Open

Client side scripts

IHDSL is a higher order programming mechanism for Smalltalk language. Higher Order Messages allow user-defined message dispatch mechanism to be expressed using an optimally compact syntax that is a natural extension of plain messaging and also have a simple conceptual model. They can be implemented without extending the base language and operate through language bridges.
        
With this technique IMPROVEKIT designed a specification language and query for measurements, a language whose syntax and semantics are similar to traditional SQL, but with a more powerful analytical level which code tends to be more concise and easy to read make this a powerful productivity tool for the end user.

An enabled user with SQL knowledge can define its own indicators and filters in IHDSL. You could build the equivalent complex SQL stored procedures in a few lines of expressive code. If you part of examples, does not need to know Smalltalk. If you know basics of Smalltalk, it opens up a new world of productivity with IHDSL.

Examples:

1. data average defects

2. data distinct type

3. data groupBy  field: #project as: 'p'  select: [:r| r count] as: 'items'

1. data select defaults

2. data where releaseName: '*2012*'

3. (data releaseName where: '*2012*') distinct type 

The following example implements a complex algorithm whose implementation in SQL not only demanded more lines of code, some far from a declarative level, a code harder to understand that IHDSL:

        
| activities rows ranks |
    activities := (ranks := ((data groupBy field: #project field: #type) select
                        field: #project  field: #type  field: #releaseName
                        field: [:group | group count]  as: 'items')
                        where: #type  isTop: 5  rankedBy: #items)

indexedOn: {#project. [:row | row quarterly]. #type}.
    rows := (data groupBy
                field: #project  as: 'project'
                field: [:row | row quarterly] as: 'period'
                field: #type  as: 'type'
                select: [:group | group count]  as: 'total') select
                field: #project  as: 'project'
                field: #type  as: 'period'
                field: #type  as: 'type'
                field: [:group | (activities join: group on: #(#project #period #type ))
                        ifNotNil: [:row | row items]]  as: 'items'
                field: #total  as: 'total'.
    rows
        transposedAt: #type
        in: ranks distinct type
        sum: #total.        

In these examples data is a variable that points to a data source (data sources are result sets, tables, queries, cubes, which reside in memory, and are loaded on demand or in batch through the File menu). The syntax is standard Smalltalk (actually the code is Smalltalk and can be inspected at runtime, debugging with the debugger, etc.) basically: 

     object message

Messages can be unary (as distinct), or arguments (like field: as :). Some commands allow data fields to refer to you by name (for example where project: 'SPI')

Fig. 46. Example of a query language and result in IHDSL

Datasources

Source data is imported full memory. It can be a table, or a "query"

Built-in datasource named Jira Issues by Zeppelin plugin, with code

Server side scripts

We call the Rubby version of our IHDSL domain language Ruby Query Language (RQL).
The same protocols and objects are available as in IHDSL, that is:

• ResultSets (rows, grouping)

• Indexes

• Statistical and mathematical functions

• SQL type commands: select, where, group by, etc:

It can be translated relatively easily from IHDSL to RQL:

translates to the following RQL code

    Ruby Query Language commands:

    select :all |:defaults | :fieldI, ... lambdaI | :field1 | lambda1 => :alias1, ... , fieldN | lambdaN => aliasN
    where field | lambda or where field | lambda => value
        Examples: where :createdYesterday or where :created => Date.yesterday or where lambda { |r| r.before? :created, Date.today }
    sort_by | order_by :field1, ... , fieldN [descending = false]
    group_by :defaults (^1) |:weekly_defaults, :monthly_defaults, :quarter_defaults | plus *_with_meta... | :field1 => :alias1, ... , fieldN => aliasN | lambdaN => :aliasN
        Example: data.group_by :project => :prj, lambda { |group| group.size} => :count
    where_ranked field, rankedByField, number
        Example: data.where_ranked :type, :items, 5
    omit_all | omit field
    distinct field
    values_of field
    result set protocol:
        join set [fields = nil]. Example: activities.join group, [:project, :period, :type]. result.join set, indexed_on field
        indexed_on field. Example: index = data.indexed_on: [:project, lambda {:row | row quarterly}, :type ]
        exists? another_set
        num field statistics: sum, average, accum_sum, accum_average, max, min, median, mode, variance, standard_deviation
         rescaled_range, gini, buckets, centiles, quartiles, ranks, moving_range, sigmas, kurtosis, skewness
    row protocol:
        join row, in field, values, accum_sum field, scalar
        is_null? field, is_nothing? field, is_zero? num_field, is_today? date_field
        equal_to? field, after? field,value, before? field, value, between field,value
    
(^1):
Defaults ||= { 
:defaults => {:project => :project, :release_name => :release_name, :release_date => :release_date } ,
:defaults_with_meta => {:project => :project, :release_name => :release_name, :release_date => :release_date, 
lambda {|r| r.instruments } => :instruments, lambda {|r| r.operations } => :operations, lambda {|r| r.actors } => :actors } ,
:quarter_defaults => {:project_category => :project, :release_quarter_name => :release_name, :release_quarter_date => :release_date },
:quarter_defaults_with_meta => {:project_category => :project, :release_quarter_name => :release_name, :release_quarter_date => :release_date, 
lambda {|r| r.instruments } => :instruments, lambda {|r| r.operations } => :operations, lambda {|r| r.actors } => :actors } ,
:monthly_defaults => {:project_category => :project, :release_month_name => :release_name, :release_month_date => :release_date },
:monthly_defaults_with_meta => {:project_category => :project, :release_month_name => :release_name, :release_month_date => :release_date, 
lambda {|r| r.instruments } => :instruments, lambda {|r| r.operations } => :operations, lambda {|r| r.actors } => :actors } ,
:weekly_defaults => {:project_category => :project, :release_week_name => :release_name, :release_week_date => :release_date },
:weekly_defaults_with_meta => {:project_category => :project, :release_week_name => :release_name, :release_week_date => :release_date, 
lambda {|r| r.instruments } => :instruments, lambda {|r| r.operations } => :operations, lambda {|r| r.actors } => :actors } 
}                
The language is integrated into the KPIs of the Zeppelin plugin, the code is a valid Ruby expression by example:

((((data.group_by :project_category => :project, :release_month_name => :release_name, :release_month_date => :release_date, 
:issuetype => :issuetype,
lambda { |group| group.average :duration } => :value)).where lambda {|r| r.after? :value, configuration["VARIABLE.TASK_DURATION"].first.to_f }) 
.transposed_average :issuetype)
.omit :value

Note the following expressions:

    lambda {| args | expression}: this is the way to introduce dynamic blocks ([] in IHDSL)
    : field_name =>: alias_name: is the way to list fields / aliases (fields: alias: in IHDSL)
    configuration ["key"]. first: is a hashmap to access the Maps of the Zeppelin Configuration

See more examples in the KPI codes

Script bundles

The scripts library, or bundles, are IHDSL scripts stored in the bundles folder of the application and can be accessed directly from the bundles menu of the main menu bar.

Simply select the menu scripts to execute useful queries or statements immediately. Note that the script may or may not display a result (ResultSet/s or graph/s), in fact they are Smalltalk / IHDSL statements.

You can edit the script by simultaneously pressing the SHIFT key. Comment your scripts to view the comment when selecting each item.

The menu has two sections separated by a line. The first is the scripts you have added to the library. The second contains smart category submenues and IHDSL filters that you can also execute or edit by directly accessing them

Fig. 47. Scripts bundles access menu