DEX Guide
Main Navigation
DEX Users Guide
About DEX - the Data Explorer
DEX is our free data explorer program designed to allow exploration of multiple large data sets from multiple types of sources, such as CSV, Tab-delimited text files, and .tensor binary data files.
They may be stored on your local file system, the internet, or AWS S3 buckets. DEX currently does not provide editing of the data set contents. This trade-off is necessary to provide very rapid access when viewing the data.
DEX is designed to load and rapidly access very large files. Where Excel or other programs will only handle about a million cells, DEX is limited only by available memory. While a file loads much faster than it would in Excel, it can still take a bit of time. However, once loaded, you can scroll over literally millions of rows or columns as fast as you can move your mouse.
DEX will also load very large binary .tensor files with float or double data. Unlike Excel, which might display 1.3845692 as 1.38, DEX displays all significant digits of content, and the full exact text from CSV files.
There are upper-limits on performance. When the entire data set does not fit in memory, it must use paging. This does allow viewing very large data-sets, but there can be remote data access waits.
Cell Features
Cell backgrounds are colored based on the data characteristics of the column they are in:
- Yellow column cells indicate that one or more cells in a column is empty
- Gray columns indicate that not all cells contain numeric data
- White columns indicate that all cells in that column are non-blank, non-empty, and all have numeric data
This can be very helpful when validating a dataset that has many rows.
Multiple Dataset Support
The OPEN menu or script command allows opening several datasets at once, each displayed in its own tab. You can also open a file by dragging and dropping it on the DEX window. You can start DEX by dropping a file on the DEX icon on the desktop.
- Right-clicking on a document's tab displays a menu of commands specific to that document.
- Clicking on the main menu displays items that apply to the entire program.
Tab Page Menu Commands
You can right-click on the tab for any open dataset. This will give you a popup menu of actions you can take for that dataset.
GO to row and column
Enter a row and column number and the display will scroll to position that cell as the upper-left displayed data cell.
PROPERTIES
This will display an information dialog with the particulars about the dataset, such as its file name and the number of rows and columns, if a CSV file has a header as its first row, and if a tensor is stored in COVARIATE or OBSERVATION major order.
SHOW INDEX
If the data store as row labels, this will toggle the left row between displaying labels and row numbers.
DOCCLOSE
This will close the specific tab page and its associated dataset.
Amazon Web Services S3 Access
The OPENAWS menu or script command displays a dialog allowing choice of AWS account/buckets and objects within the selected bucket. If no accounts exist, the lists will be empty. Selecting an account object will display the objects stored in that AWS bucket. Clicking on an object will close the dialog and attempt to load the specified object as a CSV/Tab-delimited file. An alert will be displayed if the object is not one of these valid formats.
Setting up AWS account objects on your system
Access objects are stored in the /User/your_user_name/.aws/ directory on Windows, and
the ~/.aws/ directory on Linux. Each bucket to be accessed needs an account object,
which is a plain text file with these four lines minimum:
ACCESS_KEY_ID=your_AIM_user_id_key
SECRET_ACCESS_KEY_ID=your_AIM_user_secret_key
BUCKET=the_bucket_name
REGION=us-west-2 etc.
The name of this file will be the name displayed for that account in the OPENAWS dialog.