The Portable CMPR program

The most recent additions are noted with , while noteworthy revisions added in the last few revisions are marked with .

CMPR is a multipurpose program that can be used for displaying diffraction data, pattern indexing and peak fitting and other nifty stuff. I started writing this program as a replacement for a VAX program that I also called CMPR, but it has evolved to have quite different features. I consider CMPR to be my "swiss army knife" for diffraction. Any useful manipulation that I need will probably end up here eventually. Other people have also started adding their own modules to CMPR. I will gladly accept contributed code to add new features to CMPR. If you want to do this but need some help getting started, please contact me.

CMPR requires the Tcl/Tk and BLT packages. Starting with this version, CMPR no longer uses Tix for fancy GUI features, instead it uses the BWidget package. The BWidget package is included in the CMPR code, so it does not need to be installed within Tcl/Tk.

CMPR is developed and tested on UNIX computers (SGI and Linux). It also runs on computers with the Windows-95 & Windows-NT (V4.0) and Windows-98 operating systems, but I do little Windows testing myself. It will also probably run on Macintosh computers, but it will require some work on porting parts of the BLT package as well as some AppleScript work. (Any volunteers?)

The program incorporates several small FORTRAN programs containing routines borrowed (mostly with permission) from lots of sources:

• The reflection generation code traces back to the NIST AIDS*83 (C. Hubbard, J. Stalick, A. Mighell & others),
• space group extinctions from NRC symmetry codes (A. C. Larson),
• peak fitting uses GPLSFTA (D. Cox, W. Hamilton, L. Finger & many others),
• Gaussian smoothing/peak search routines were supplied by R. Harlow and A. McGhie.
• the ITO autoindexing program was written by Jan W. Visser [J. Appl. Cryst. 2,89 (1969)], with contributions from Robin Shirley and Jerry G. Johnson, Jr.
• the DICVOL autoindexing program was written by A. Boultif and D. Louër, [J. Appl. Cryst. 24, 987-993 (1991)]
• the TREOR autoindexing program was written by Per-Erik Werner [J. Appl. Cryst. 18, 367-370 (1985)]. Note that a newer version of TREOR is due as part of the EXPO package soon, look for "Indexing by N-TREOR in EXPO" by C. Giacovazzo et al. in J. Appl. Cryst..

Mouse actions

Zoom

It is possible to enlarge the size of the plot window by dragging the frame of (the border surrounding) the plot window. The scale of the plot will adapt to show the same range of data, but now expanded to fit in the larger window. It is possible to change scale by "zooming in" and display a smaller region of the data in the plot window by clicking in two opposite corners of the region to be shown with the left mouse button (a left-click) (examples: PLOT before and PLOT after). Zooming in can be done multiple times. It is also possible to "zoom out" and show the previous view of the data by clicking on the right mouse button (a right-click).

Pressing the "z" or "Z" key (Z for Zoom) allows the zoom to be set manually by entering minimum and maximum values for the x- and y-axes.

Live Cursor

Pressing the "l" or "L" key (L for Live cursor) in the plot window creates a panel below the plot that shows the coordinates of the mouse. Pressing "l" or "L" again, or pressing the "Close cursor display" button causes this panel to be removed.

Buffer Selection

Most modes within CMPR display a list of datasets that have been created or read in. I call these in-memory datasets buffers.

• left-click Buffers are selected by clicking on them with the left mouse button
• control-left-click The control and shift keys can be used to modify which buffers are selected. Holding the control key down and clicking the left mouse button causes the buffer to be selected or unselected.
• shift-left-drag Holding the shift key down and dragging the left mouse button across part of the list of buffers causes those buffers to be selected.
• double-left click causes the current buffer(s) to be plotted (can be combined with shift or control).
• triple-left click or a right-click (single) causes all buffers to be selected and plotted.

Deleting Buffers

Control-right-click causes the selected buffers to be deleted from the program (not from disk). You are prompted to confirm before any buffers are deleted.

Reflection Labeling

In the EditCell section of the program, it is possible to label the generated reflection position.
• pressing the "H" key: labels individual reflections (also Shift key+left mouse click)
• pressing the "A" key: labels all reflections (also shift+left double-click)
• pressing the "D" key: deletes all reflection labels (also shift+left-doubleclick). Note that individual reflection labels will be typically be erased automatically after a given number of seconds, as set in the Display options page.

Keyboard Shortcuts

There are a number of shortcuts implemented by pressing keys or mouse buttons. Keys are not case-sensitive. in plot window/any page
• L -- shows "live" cursor position
• Z -- set zoom scaling manually
• left click -- zoom in
• right click -- zoom out
in plot window/EditCell page only
• shift+left click -- label nearby reflections
• shift+double left click -- label all reflections
• shift+right click -- delete reflection labels
in plot window/fit page only
• control+double left click -- add peak at current position
• S -- set fit range to current zoom
• F -- scroll the fit region forward
• B -- scroll the fit region backward
• E -- expand the scroll fit region
in any window/any page:
• control+C -- exits the program
• F1 key -- show help on current page
Also see Buffer Selection tricks

Read (GUI)

This page is used to read diffraction data in a number of formats.

To read a file into a buffer, select the format on the right and then select a file on either the window that pops up (Windows) or on the central pane (UNIX). Note that files can be read either by double-clicking (with the left button) on them or by pressing the Open (or Read) button.

After a file has been read, it appears in the plot window and if the legend is active (see the Display options page) the name appears in the legend. Also see the buffer selection section to see how one or more buffers can be plotted.

It is quite easy to extend CMPR to support additional formats as well. (No compiler is needed). For information on creating a new routine for reading data, see the Customizing section, below.

Write (GUI)

This causes the selected buffer(s) to be written. The box at the top shows the directory where files will be written. Note that forward slash characters (/) are used, even for Win-95 and -NT.

It is quite easy to extend CMPR to support writing additional formats as well. (No compiler is needed). For information on creating a new routine for writing data, see the Customizing section, below.

Plot (GUI and PLOT)

The plot page is used to display one or more buffers. To select a single buffer, click on the buffer name. A range of buffers can be selected by dragging with the mouse. Individual buffers can be selected or deselected by holding the control key and clicking on the buffer name. The plot is updated when the "Update Plot" button is pressed or with a mouse double-click.

The "Postscript out" button causes a postscript copy of the plot window to be created. Settings on the Display options page determine where the file is created or how it is printed.

Plots can be exported to grace, a publication-quality graphics package where the displayed plot can be enchanced and annotated. This works very nicely.

This page is also used to change the way a file is displayed. This is done by selecting the file(s) to be changed and then using the appropriate checkboxes to select the color, line type, symbol type and size. The changes will be made when the "Apply Changes" button is pressed.

Rescale (GUI)

This page is used to modify the stored values in one or more buffers. One can specify constants to multiply the x and y values and constants to add to the x and y values (addition is done after multiplication).

It is also possible to transform the x and y scales. The x-axis can be transformed between Q, d-space and 2-theta units for an arbitrary wavelength. Intensities (y values) can be transformed to square roots [Sqrt(I)], base 10 logarithms [Log(I)]. The I/sig(I) mode divides each intensity by its estimated error, providing an estimate of the signal-to-noise and the I2/sig(I)2 mode scales each intensity value so that that estimated error is the square-root of the intensity, thus showing raw counts prior to scaling (or the equivalent).

Conversion is done before constants are added or multiplied.

Changes are applied when the "Rescale" button is pressed. If more than one scaling or transformation is applied, the operation is applied to the results from the previous operation. However, if the "Reset previously applied scaling" checkbox is selected, the x and y values are reset to the values that were read in, before any scaling or transformation is done.

Combine (GUI)

The Combine page is used to create a new buffer set by adding or subtracting up to five other buffers. The buffers must have the same number of data points. No attempt is made to offset buffers, so they must all start (and end) with the same x value.

HKLGEN (GUI)

This page generates a list of the allowed reflections for a given set of unit cell constants and extinction conditions. It is possible to specify symmetry by space group or by selecting individual translational symmetry operations.

Space groups should be entered as specified in GSAS: Enter the centering condition or cell type (P, R, A, B, C, I or F) and then the symmetry along each appropriate axis, with spaces between each axis. Append a final "R" for rhombohedral settings. Some examples are: "P 1 21/c 1" or "P 21/c" for P2₁/c and "R 3 m" and "R 3 m R" for the hexagonal and rhombohedral settings of R3m, respectively. If an incorrect space group is entered, no error message is generated, but no extinctions are generated.

Unit cells may be input with either direct or reciprocal cell parameters. The reflections are generated and placed in a separate window when the "Compute" button is pressed. This table can be saved to a disk file by entering a file name and pressing the "Save as" button.

EditCell (GUI or PLOT)

This page generates a stick-figure plot of the allowed reflections for a given set of unit cell constants. Reflections can be omitted or highlighted if extinct due to symmetry.

Unit cells may be input with either direct or reciprocal cell parameters. Use of reciprocal cell parameters has proven valuable for indexing powder diffraction data with low symmetry unit cells where some reciprocal unit cell vectors are known from electron diffraction.

Symmetry can be specifed by space group or by selecting individual translational symmetry operations. Space groups are entered as used in GSAS. Enter the centering condition and then the symmetry along each appropriate axis, with spaces between each axis. Append a final "R" for rhombohedral settings. Some examples are: "P 1 21/c 1" or "P 21/c" for P2₁/c and "R 3 m" and "R 3 m R" for the hexagonal and rhombohedral settings of R3m, respectively. If an incorrect space group is entered, no error message is generated, but no extinctions are generated.

After the unit cell parameters have been entered and the "Compute" button is pressed, reflections are generated and displayed as a stick figure. At this point list of buffers is then displayed. If symmetry information was entered before the compute button was pressed, the extinct reflections are not included in the reflection list. If symmetry information is entered after the compute button is pressed, the extinct reflections are highlighted in yellow. To superimpose the stick figure on one or more buffers, select the buffers and press "Update Plot" button.

The plot will respond to changes in the unit cell parameters or zero correction made either by typing value(s) into the entry boxes or by sliding the bars to the left of each number. The zero correction is added to two-theta values. Unit cell symmetry can be raised by selecting a higher symmetry cell from the list in the middle. To lower the symmetry, a new peaklist must be created.

To redisplay a previously generated set of reflections, select the appropriate buffer using the "Select peaklist" button in the upper left of the page. Note that this recenters the unit cell sliders and recomputes the reflections within the requested angular range (see Display options).

Individual reflections can be labeled by holding the Shift key and clicking on the reflection position with the left mouse button (shift-left-click). All reflections can be labeled by holding Shift key and double-clicking anywhere in the plot with the left mouse button (shift-left-doubleclick). Individual reflection labels will be typically be erased automatically after a given number of seconds, set in the Display options page; an erase time of 0 means that reflections labels are not automatically erased. Holding the Shift key and click anywhere with the right mouse button (shift-right-click) clears labels for all reflections, The size of reflection labels can be changed on the Display options page. (see Display options).

Note that the size for the reflection markers, with respect to the Y axis, is determined by the "Maximum and Minimum Y values" on the right side of the screen.

In the example images, (GUI or PLOT), the reflection list was generated using space group R -3 but then space extinctions for R -3c are labeled. Thus, the reflections made extinct by the c-glide are shown in yellow. Reflections extinct in R -3 are not shown.

Fit (GUI or PLOT)

Fit is used for profile fitting of selected regions of a pattern. Peaks are fit with a pseudo-Voigt where eta is the "mixing term." A value of 0.0 for eta selects a completely Gaussian peak shape and a value of 1.0 selects a completely Lorenztian peak shape.

To fit peaks:

1. A buffer to fit is selected using the "Select Dataset" button in the upper left. If only one appropriate dataset is present, it is selected automatically.
2. A buffer to record the peak positions is selected. The first peak list (if any) is the default, but it is also possible select "new" which will add peaks to an empty list.
3. Select a data range to be fit. The range must contain less than 1000 data points. The range can be set using the mouse to zoom and the S/Z/F/B/E keyboard shortcuts or use the "Set range to fit" submenu. This creates a dialog box (GUI) to set the data range with three options:
   • use the mouse to zoom in on a region and press the "Set from zoom" button, or
   • type an upper and lower two-theta value in the "Fit Range" boxes, or
   • shift the range using an increment and "Add increment to limits" button.
   When a range of data is selected, any peaks found in the peakfile in that data range are displayed on the plot and are added to the peak list menu to the right.
4. There are two methods for adding new peaks to the menu:
   • by pressing the "P" key (P for Peak) while positioning the mouse to the location of the peak maximum (shift+double-left-click also works)
   • by pressing the "set" button in any row of the peak table and then pressing the "P" key, after moving the desired peak maximum. This "set" button can also be used to redefine peaks in the table.
5. Select parameters to refine: Once at least one peak has been entered, and a suitable range has been selected, a "Run GPLSFT" button appears. Select the parameters to be refined by clicking on the appropriate boxes to the right of the parameters
6. Optimize parameters by pressing the "Run GPLSFT" button. The least-squares algorithm in the program is far from robust. It is wise to first refine the peak areas and FWHM terms and then add background and eta and peak positions after reasonable fits are obtained. If the fit blows uo, refine fewer parameters at a time or decrease the damping value.

The GPLSFT program can use the Finger-Cox-Jephcoat correction for peak asymmetry. For this correction, three terms must be known: the diffractometer radius, the half-height (or half-width) of the sample and of the detector. Values may be specified in any consistent units. (NIST would prefer that you use metric units, of course.) While these values can be refined, do so with care: do not refine too many variables at the same time, use a small number of cycles and set the damping factor to a small number.

By default fit results are shown superimposed on the plot of the fit, but the results can be placed in a separate window by deselecting the "Report fit results in plot window" checkbox on the Display options page.

Peaks/Smooth (GUI)

This page is used to smooth one or more diffraction patterns with a Gaussian. The smoothed results can be used to compute a first and second derivative of the data as well. Peak positions, as well as height and area estimates can be made using the second derivative. Once the peak positions are determined, sections of data away from the peak positions are used to construct an estimate of the background.

The "FWHM" setting controls the width of the Gaussian convolution function for all of the above computations. It seems that it is best to set this to the approximate FWHM seen for peaks in the data.

The "Sensitivity" and "ESD" settings are used only for peak searching. Both are used to reject peaks below signal-to-noise criteria, but the exact functioning of these settings is not well understood.

This code is still under development and comments and suggestions can be offered to Dick Harlow.

Index (GUI or Results Window)

This page is used to prepare input to three autoinexing programs, ITO, TREOR and DICVOL. These programs attempt to find unit cells that index a set of diffraction peaks.

The list of peaks is compiled into a buffer by using Peaks and/or using Fit (Fit is recommended for the most accurate peak positions). The peak list is then selected with the button on the upper left. A list of peaks is then shown to the right, where it is possible to select which peaks are used for autoindexing. Input files are created for all that are selected via the checkboxes to the left when the "Write file(s) & Run" button is pressed. For each selected program, it is then possible to select that the program be run in "foreground" (CMPR pauses while the run progresses) or in "background" (CMPR continues). Note that values for various program options are provided in the center section of the this page. Defaults values are used for all program options. However, it is possible to override these values from this section. For example, on the GUI example, the maximum cell length allowed by TREOR is increased to 30 Angstrom.

Once any of the programs have been run, the output file can be examined using the "Review Output" button. Two windows are created: the first shows the output file from the indexing program in a scrollable window and the second window shows the unit cells that have been located. Pressing the "Set" button to the left of any cell parameters causes those values to be loaded into EditCell. This allows the unit cell to be compared to an observed diffraction pattern.

Options

The Options entry on the menu bar is used to change the appearance of the program on your screen. By selecting the "Display Options" menu item, a window is created where many options within the program can be selected, see below. The "Screen Font" menu item can be used to change the sizes of fonts used throughout the program. (See below for information on how to modify the default value for the screen font).

Selecting the "Display Options" menu item opens a window used to set values used in other parts of the program (click here to view). The sections within this window are:

"Plot options"

determine how data are displayed and recorded

• "Plot legend" determines if a legend is shown on the right-hand side of the plot.
• "Write PostScript" and "Print PostScript" options determine what is done when the the "PostScript out" button is pressed in sections of the program, such as Fit and Plot.
• If "Write" is selected, a file name is specified as the "Postscript file name".
• If "Print" is selected, a UNIX command for processing the file is specified as the "Command to print files" (this option is not available in Windows).

"Peak Fit Options"

are used for the Fit section of the program.

• The "Report fit results in plot window" option controls if the fitted parameters are shown in the plot window or in a separate window.

"EditCell options"

are used in the EditCell section of the program.

• "2theta max" determines the maximum hkl value computed by the program
• "max change in cell parameter" controls the range and thus the sensitivity of the sliders
• "HKL label erase time" specifies the number of seconds before reflection labels are erased in seconds. A value of 0 or less means that these labels are not automatically erased.
• "HKL label size" specifies the size of the reflection labels in pixels (or points for negative values).

"Help Options"

determine how help information is displayed. Help can be displayed in a browser (Netscape must installed for UNIX systems) or in a help window.

Help (GUI)

Help can be used to find out more about a section of CMPR. Help can be displayed on a web browser or in a separate window, depending on the setting on the Display options page. The web help is more complete, but may be more cumbersome.

Help is invoked by pressing the F1 key or using "Help on current page" in the Help section of the menubar. The help is somewhat context-sensitive help as the section of help information relevant to the current page is displayed.

Initialization files

CMPR reads default values by sourcing file localcmds.tcl from the directory where CMPR is installed and then file .cmpr_init from the user's home directory (or C:\.cmpr_init in windows). There are no errors if either or both files do not exist. Various Tcl commands can be placed in this file. One example is:

set graph(font) 18

This command overrides the default value for the screen font from 14 point (set in the cmpr.tcl file) to 18 point.

Many other options can be modified in this fashion. Users are referred to the CMPR code to find them and see how they work.

Reading directly from GSAS experiments

The read_gsasexp.tcl file reads diffraction data (observed and computed) as well as reflection positions from the GSAS .EXP, .Pxx and .Rxx files. To do this, it must locate the directory containing the GSAS programs tcldump or hstdmp. This is done by searching the directory defined by environment variable GSAS (if defined) and then sequentially the directory below where cmpr is loaded (../gsas), /usr/local/gsas/exe, c:\gsas\exe and ~/gsas/exe. If you do not see a checkbox labeled "GSAS EXP file" for a file format, define the environment variable GSAS to define the location where GSAS is stored.

Using the Example File Formats

When CMPR is downloaded, there are a fair number of file formats available for use. If you would rather not have these formats appear on the screen, you can rename formats you do not wish to have to a name that does not begin "read_*.tcl" (for example "Example_read_*.tcl" or "removed_read_*.tcl").

Supporting Additional File Formats

To define new formats for reading a file create a file called read_????.tcl where you supply a name of your choice for '????'. Likewise, to support a new output format, create a write_????.tcl file. Note that formats are listed on the menu alphabetically by file name. Follow one of the other read_*.tcl or write_*.tcl files as an example. Place the file in the same directory as the cmpr.tcl file (and the other .tcl files).

Notes on coding a read_????.tcl file

1. Define a commandline option to read a file (optional) using command(cmdopt) and defining a routine associated with the command option for example:

   	lappend command(cmdopt) -xxx
   	set command(-xxx) readxxxdata
   

   defines that command

   	cmpr.tcl -xxx yyy.xxx
   

   can be used to read file yyy.xxx using read routine readxxxdata (see below)

   You can define as many options as you want or none, but be sure to define them in pairs, as above. If you don't want to support reading this type of file from the command line, you don't need to include these two statements.

2. Define the dialog access define the text to be shown

   	lappend command(readtypes) "LHPM/RIET7"
   

   define a command to read the data

   	lappend command(readproc) ReadLHPM
   

   define a list of allowed datatype suffixes

           lappend command(filterlist) {dat}
   

   If you will have upper and lower case versions for Unix, you might want to do the following:

   	if {$tcl_platform(platform) == "windows"} {
   	    lappend command(filterlist) {dat}
   	} else {
   	    lappend command(filterlist) {dat DAT}
   	}
   

   Define a datatype listing (not used at present) to describe the suffix:

   	set command(ReadLHPM_dat_type) "LHPM/RIET7 data"
   	set command(ReadLHPM_DAT_type) "LHPM/RIET7 data"
   

   Note that the info in parentheses corresponds to the name used for command(readproc) followed by each value used for command(filterlist).

3. Define a dialog box read routine This almost always looks the same, except for the readXXXXdata line which refers to the real proc used to read the data.

   	proc ReadLHPM {file} {
   	    global command
   	    if {$file == ""} return
   	    pleasewait "reading file $file"
   	    set ret [readLHPMdata $file]
   	    donewait
   	    if {$ret != ""} {return $ret}
   	    showlastentry $command(read_filelist)
   	}
   

4. Define a routine to actually read the data. Try to imagine how the routine will crash and burn when a user specifies a data file with the wrong format.

Look at the other read routines to see examples.

The CMPR program will automatically include the new format the next time it is started. Likewise, to remove a format all that need be done is to change the file name so that it starts with something other than read_ or end with something other than .tcl, for example note the files named Example_read_*.tcl.

Notes on coding a write_????.tcl file

1. add the format name to command(writetypes) and the procedure name to command(writeproc)

   	lappend command(writetypes) ".csv (Spreadsheet)"
   	lappend command(writeproc) writelistcsv
   

2. Write a proc to write the data/peaks Note that you can use the statement

   	if {[set ${data}(type)] == "peaks"} {
   

   to take different action depending on if a peak file is being written Likewise, you can use the statement

   	if {$command(writeunits) == 0} {
   

   to take different action if scaled values or original values should be written.

Look at the other write routines to see examples.

The CMPR program will automatically include the new format the next time it is started. Likewise, to remove a format all that need be done is to change the file name so that it starts with something other than write_ or end with something other than .tcl.

Comments, corrections or questions: crystal@NIST.gov

$Revision: 1.6 $ $Date: 2001/11/09 19:01:47 $