Data access. Pretty mundane, wouldn’t you say? It’s … just there, doing its stuff, while you worry about how to best present the sea of data so that your users can glean the most important information from it. Well, in order to make your job even easier, we’ve made some major changes to our data access components in v15.1. Are you ready?
Query Builder improvements
It was in v14.1 that we first introduced our Data Access library to our product line. This library provides a set of tools intended to simplify the process of binding to different kinds of data sources.
Among this set of tools, I think the Query Builder reigns supreme. It's used by several wizards to provide you and (or?) your end users with a visual interface for constructing custom SQL queries. Unfortunately the Query Designer had some notable limitations from the outset (for example, it did not cover such SQL features as sorting and grouping).
To address this issue, we have made several improvements in v15.1 to the Query Builder’s functionality and usability, without forgetting increasing performance where necessary.
New features:
- A columns list that allows you to preview the set of table columns before adding a table to a query.
- Revised dialog interface.
- Improved performance. Column information is now loaded on demand for each data table separately. The performance increase will be especially notable when binding to databases containing many tables with a massive number of columns (for example, when binding to Teradata – see below for information on this).
- Support for aggregate functions.
- Support for sorting. If the resulting data is sorted by more than one column, the order in which sorting is applied can be adjusted manually. Specified sorting settings correspond to the optional ORDER BY section of the resulting SQL string.
- Support for grouping. Specified grouping settings correspond to the GROUP BY section of the resulting query string.
Adding the ability to sort, group and aggregate the data provides more comprehensive coverage of the data shaping functionality of SQL. When you apply such a shaping to a field, it will be marked with a corresponding icon, and this makes the whole process of shaping data more visual and intuitive.
Improved memory consumption
For 15.1 we have improved the internal structure of our SqlDataSource class, as well as the algorithms we use to obtain data from a database table. The upshot of this work is that we’ve drastically reduced its memory consumption in real-life scenarios.
To see the difference between the memory consumption of a SqlDataSource object (bound to the same database) between versions 15.1 and 14.2 , take a look at the chart below. This shows the aggregated profiling information we obtained as provided by the dotMemory profiler. This chart illustrates the dynamic aspect of memory allocation by SqlDataSource from the beginning to the end of the data loading process.
The raw profiling data was obtained from loading the same 3.5-million-row data table, consisting of 20 columns of differing data types.
As you can see, the memory usage has been reduced in v15.1 by about 65%. The improvement in memory consumption is even more pronounced when binding to data tables that contain mostly columns of numeric data types.
Teradata support
With v15.1 we have added support for Teradata (version 13 or later) to our Data Access library. Teradata is a massively parallel processing database management system that typically operates on extremely large amounts of data. Yes, you got it: we’re suddenly talking Big Data.
You can now find Teradata in the list of available data providers in the Data Source Wizard when creating a SQL data source. After that, the process of setting up a data connection will be the same as for any other SQL data provider: you need to specify the server address, the required database name, and the user credentials.
After a connection is set up, you can go to the next wizard page and construct an SQL query to obtain data from tables in the database. All the connection magic is hidden; you just use it as any other SQL data provider.