Visualization of Time-based Multimodal Data
This guide provides an overview of some of the capabilities of ChronoViz.
A more detailed manual is forthcoming.
- Before You Start
- Basic Functionality
- Feature Details
ChronoViz requires a computer running Mac OS X 10.6 or later.
To install ChronoViz, simply unzip the downloaded file. The ChronoViz application can be placed anywhere on your computer, but it is recommended that you move it to the “Applications” folder.
Analysis with ChronoViz usually begins by opening a video (See Note 1 below). To open a video, go to the File menu and select “Open…”. In the window that pops up, you can select any video file that Quicktime understands (See Note 2 below), or an existing ChronoViz “Annotation File.”
Once you select a video file, the video will be displayed in the main ChronoViz window, which consists of the movie viewer, a timeline, and controls for moving in time. The current time is indicated as a graphic indicator on the timeline and as a timecode below the “Play” button.
- You can also begin analysis by selecting data (instead of a video). See the “Importing and Aligning Data” section below.
- For compatibility with a broad range of internet video formats, such as DIVX and FLV, it is recommended that you install Perian (http://perian.org/). There are currently some known issues with playing Windows Media files, so it is recommended that you convert these files to another format before using them in ChronoViz.
There are several ways you can move the current time:
- Normal playback: Click the “Play” button to playback at a constant rate, and the “Pause” button to stop. The playback rate can be set by selecting “Preferences…” under the “ChronoViz” menu.
- Dragging time indicator: Clicking and dragging anywhere there is not a visible annotation on the timeline will move the time to the point indicated by the cursor position.
- Stepping: When the main ChronoViz window is active, the arrow keys can be used to step in specified increments. The Right and Left keys will move one frame forward and backward, respectively, and the Down and Up keys will move by a specified step size that can be set in the Preferences.
- Jump to annotations: Clicking on any of the annotations that are displayed on the timeline will move to the beginning of that annotation.
- Other data sets: All other types of data sets allow moving the time point. Please refer to the specific details for each data set for more information.
Next to the play button, there are two buttons to configure the play mode. The first button will make it so that playback automatically repeats. The second button will make it so that playback ends when it reaches the end of the currently selected annotation (which will be the last annotation that was clicked on the timeline).
Two general types of annotations are supported in ChronoViz: “Point” annotations that only have a single time point, and “Duration” annotations that have both a start and end type.
Annotations can be associated with one or multiple categories. The first category that appears in the annotation’s category list is considered its “main” category and determines how it is displayed and aligned. Other categories can be useful for filtering and searching. Categories can be editing by selecting “Edit Categories…” under the “File” menu.
There are several ways to create a new annotation:
- Clicking on the “New Annotation” button will create a new annotation at the current time and bring up the “Annotation Inspector”, which gives full access to all of the details of the annotations.
- Pressing the “Return” key while the main ChronoViz window is active will bring up a “Quick Entry” floating window. Here, you can select the category for the annotation and enter a text description of the annotation.
- Pressing a category’s shortcut key while the main ChronoViz window is active will enter a new annotation at the current time for that category with a blank description. Holding down the shortcut key will create a “duration” annotation
There are a couple of features to be aware of with the timelines:
- Temporal Zooming:
Pressing the zoom buttons in the toolbar will increase the temporal resolution of the timeline. An overview timeline will appear beneath the main timeline, where you can move the selected time region, as well as drag the end points of the selected time region to further adjust the temporal resolution.
The vertical size of the timeline can be changed by dragging the bar at the top of the timeline. The appearance of annotations on the timeline will change depending on the amount of space available. As you increase the size, labels may appear on the annotations, and point annotations may be staggered to show more information.
- Multiple Timelines:
Additional timelines can be shown by selecting “Add Timeline” under the View menu. Each timeline can be configured to show annotations in different ways.
When you hover the mouse cursor over a timeline, a pair of buttons will appear in the upper-right part of the timeline. By clicking on the button with horizontal lines, you can rearrange the order of timelines (if there are multiple timelines being shown). By clicking on the gear icon, you will be presented with menu of options for changing the timeline.
By selecting the filters option under the configuration menu, you can filter the categories that are shown for that timeline.
Under the Window menu, there is an item labelled “Annotations Table.” This will present a table of all of the annotations. The table can be sorted by clicking on the table headers, and can be filtered by typing terms into the search field on the upper-right of the window. Clicking on any annotation in the Annotations Table will move the time point to the beginning of that annotation.
By default, the Annotation Table will show a single line with truncated information for each annotation. To show the full information for the annotations, click the “Detail” button at the top of the Annotations Table window.
In ChronoViz, data is imported in the form of data sets, which are associated with data sources. For most cases, a data source is a single file on the disk, which may contain multiple potential data sets.
To import data, you can either click on the “Add Data” button in the toolbar or select “Import Data…” from the File menu. If no other data is already imported, you will be asked to select a file, otherwise a window will open with the last data source used. To open a new data source file, click the “New Data Source…” button at the top of the window.
Once a data source is loaded, you will be presented with a list of data variables in the data source, as shown in the example below. At the left of this window are two columns of checkboxes. Active means that the data is loaded into the system but not displayed (for example, to be used with computational analysis). Check the appropriate boxes next to the data variables that you wish to import, make sure the appropriate data types are set, then click “OK” at the bottom of the window. If you'd like, you can double-click on the “Data Label” to give the data set a different name.
Depending on the data source, the options under “Data Type” may vary. Some of the possible options include:
- Time Series: The data is imported as a series of time/value data points, and displayed as a time series graph on a timeline.
- Geographic Longitude/Geographic Latitude: The data is imported as a series of time-coded geographic positions, and displayed as a path on a map. Note: Be sure to specify two data columns, one for latitude and one for longitude.
- Annotation: An annotation is created at the time specified by the time column with the text content for that data column.
In the example below, a GPS log in the form of a CSV file is the data source, and the latitude and longitude have been specified to be imported.
The most general purpose data type that is supported is CSV, or comma-separated value, files. These are text files containing data in rows and columns, with the columns separated by commas. Microsoft Excel, along with many computational analysis programs, supports reading and writing CSV files.
When you import a CSV file, you will need to specify how time is coded. There are three options:
- None: There is no time coding, and the data will be evenly distributed across the current time duration in ChronoViz.
- Real/Absolute: Times are encoded at absolute times, possibly with a Date column. If there is no Date, then times are assumed to be on the same day as the rest of the data.
- Relative: Times are in seconds (or fractions of seconds) from the beginning of the data set.
Select the appropriate option for your data set, and also make sure that the correct column is indicated for the time column.
Transcripts can be imported in the form of InqScribe files. To import an InqScribe file, either click on the “Add Data” button in the toolbar or select “Import Data…” from the File menu. After you select the InqScribe file, you will be presented with the option to select two types of data to display, which are described below. After you check the appropriate boxes, click “OK” and the data will be displayed.
Transcript Data: This presents a window of the transcript in a form similar to its appearance in InqScribe, as a page of formatted text with time codes. With this display, any of the text can be selected to move to that point in the video. This display will also follow the time point if it is moved in another way, such as from the timeline.
Annotations: This will create new annotations that correspond to each entry in the transcript. The annotations will be assigned categories that represent each speaker, and will be color-coded and aligned on the timeline.
When annotations from the transcript are first placed on the timeline, they usually appear just as colored boxes. However, if you increase the size of the timeline and zoom in, the content of the part of the transcript represented by those annotations will be visible.
ChronoViz also supports several other types of data, which will be described in more detail in a future version of this document:
- Image Sequences, displayed in the same manner as a movie.
- Digital Pen data, displayed as a page of notes and as annotations on the timeline.
These types of data can be imported from the following data types:
- Microsoft SenseCam logs
- Digital Pen data from the UCSD Ethnographer++ system. Please see the ChronoViz Digital Pen and Paper Integration Guide for more information.
Additional video and audio files can be shown and aligned. To load a secondary video or audio file, go to “Import Data…” under the File menu.
To align the new video, first find an appropriate point in the main video that will serve as a match for something in the new video. Then, click the button labelled “Adjust Alignment” underneath the new video. Clicking this button will let you move the position of the new video independently of the main video, and you should find the frame that matches the one shown in the main video. When you are done, click “Save Alignment”.
When you load in an audio file, a new timeline with an amplitude graph of the audio will appear. To align the audio, right-click (or control-click) on the amplitude graph at a point that matches the current time point shown in the main video, and select “Align to playhead”
If you already know the offset (in seconds) of an audio or video file, you can enter it by selecting “Linked Files…” under the File menu, and double-clicking on the offset number for that file.
Data in ChronoViz can be exported in a number of ways by going the the File menu and selecting one of the options under “Export”. A few are described below:
- Annotations: A table of annotations in CSV format.
- Video Clips: Individual video clips corresponding to the Duration annotations of a selected category will be extracted and saved with information about the original video and its start and end time in the original video. This information can be seen by selecting “Get Info…” in the Mac OS X Finder.
- Image File: A PNG image of one of the timelines
Plugins can be written to perform automatic data analysis. These are written in the Python programming language. Please contact me for additional information if you are interested in writing a plugin.