Aug 9, 2017

Version 3.6 is out

Here is what's new and exciting in version 3.6:

Multiple file loading

Previously, loading multiple uniform files (i.e. files with the same type and similar set of columns) required using iterations. This could be challenging for new users unfamiliar with the concept of iterations. Starting from this release, all file import transformations allow loading multiple uniform files at once without using iterations. Loaded files are automatically concatenated into one table. it makes is convenient to load several daily or monthly reports at once.

New mode for loading multiple uniform files at once.


The files to load can be specified either explicitly by ticking necessary files in list, or by providing a search criterion (substring, wildcard, or regular expression). The source folder path can be specified using a parameter.

Publishing to Tableau Server
Starting from this release the "Export to Tableau" transformation can publish Tableau data extracts right to Tableau Server as data sources.

Publishing to Tableau Server


Support for fixed width text data

We've added two transformations to deal with fixed width text: "Import fixed width text" and "Split fixed width text". The first transformation loads a text file that contains columns of fixed width. It has a preview mode for convenient "slicing".

Import fixed width text (click to zoom).

The other transformation, "Split fixed width text" allows splitting columns with fixed width text that were already loaded from other data sources (e.g. a database). It has the same preview mode for easy, visual "slicing".

7 other new transformations

In total, this release adds 9 new transformations -- more than any previous release ever.
  • Fill Right transformation is similar to the “Fill down” transformation. It fills empty cells to the right, in specified row. This transformation helps dealing with spreadsheets where table headers contain merged cells.
  • Download File transformation fetches a file over the internet using the HTTP(S) or the FTP network protocol. It supports basic login/password authorization.
  • Split Delimited Text transformation is similar to the “Import delimited text file” transformation (introduced long time ago), but splits an already loaded column into new columns. It can be used for instance for dealing with CSV data imported from a spreadsheet, or a database.
  • Status transformation is a workflow transformation. When reached it displays a custom message. Typically, it’s used for signaling on project execution progress. For desktop versions of EasyMorph the message text appears in the “Run progress” dialog window. For EasyMorph Server the message appears as task status in the web-console, and also is logged into the task log. Custom statuses of called/iterated projects pop up into their calling projects, which allows displaying iteration progress (e.g. “Processed 15 files out of 132”).
  • Regular Expression transformation find matches for specified regular expression. The transformation works in two modes: find 1st match, and find all matches. In the latter case, new rows can be inserted into the table to store matches beyond 1st. Most common use cases for the transformation include data quality checks and text parsing.
  • Select By Lookup transformation select columns which names exist in another (lookup) table. It is intended for advances scenarios in which a resulting set of columns depends on user input or calculation logic.
  • Convert Data Type transformation converts text dates in one or more columns into number dates. The date format in selected columns can be auto-detected by pressing “Detect” button, if it matches one of 1,000+ formats known to EasyMorph. If no format detected then it should be entered manually in transformation properties. The format specification is the same as for the format() function.

Changes to existing transformations

  • The Concatenate function of Aggregate now allows specifying a separator and concatenating only distinct values. 
  • Running Total transformation has got a competition ranking mode in addition to the default dense ranking. 
  • Select Matching Database Rows and Delete Matching Database Rows transformations now allow matching on dates (without times). 
  • Iterate transformation now runs faster and is more memory-efficient.

The system function

A new function has been added: system(keyword). This function allows obtaining various system values. The keyword must be a text constant. Currently supported keywords:

  • projectpath — The full path to the current project including file name.
  • projectfilename — Current project’s file name only.
  • programpath — The full path to morph.exe.
  • currentpath — The working directory.
  • user — The current user. Replaces function user().
  • callerpath — The full path of the calling project including file name.
  • utcoffset — The UTC offset to the system time zone ( e.g. "-04:00:00").
Example:
system('projectfilepath') returns ‘C:\Documents\myproject.morph’.

Custom HTML in generated documentation

Auto-documentation now allows including custom HTML into the head and body of generated HTML page. This can be used for inserting corporate logos, links to other HTML pages, change logs, custom project metadata.

Miscellaneous

  • The visual condition builder in Query Editor now has undo/redo buttons. 
  • Added support for SAP HANA SQL dialect (for ODBC connections). 
  • Query preview now have a context menu invoked by right-clicking on datagrid cells. 
  • Expression editor has got a button for inserting date constants. 
  • Project tabs can be switched back and forth by pressing Ctrl+Tab and Ctrl+Shift+Tab
  • The “Input” transformation can now be enabled in Plus license as an add-on feature.