The Portable CMPR & LOGIC programs

Updated December 2004

Introduction

CMPR is a multipurpose program that can be used for displaying diffraction data, manual- & auto-indexing, peak fitting and other nifty stuff. I started writing this program as a replacement for a program by the same name that was part of my VAX Powder Suite package. This version of CMPR has evolved to have many features not present in the VAX program. I consider CMPR to be my "swiss army knife" for diffraction. Any useful manipulation that I need that does belong anywhere else will probably end up in CMPR.

The LOGIC program also dates back to the Powder Suite package. It is used to locate entries in the ICDD-JCPDS PDF-2 powder diffraction database that match specifed conditions, i.e. contain certain elements and not others, have peaks in certain locations, but do not have peaks in certain other locations. The database must be licensed from the International Centre for Diffraction Data for every computer where the database will be used. The VAX LOGIC program originated the concept of full Boolean logic searching of multiple database fields, which is now the basis for the ICDD PDF-4 product. Unlike the ICDD's PDF-4 product, LOGIC allows searching on only a very limited number of fields: chemical composition, number of unique elements, peak positions & intensities, subfiles and entry number, but logical constraints on these fields may be "mixed & matched." Entries can also be located that by matching text strings. The capabilities of LOGIC can also be accessed with in CMPR, which allows database patterns to be plotted along with experimental data.

Note that if access to the ICDD database is not purchased, the LOGIC program can be run only with test data. No such requirements apply to the CMPR program, which is freely distributed.

Both CMPR and LOGIC should run on most common computer platforms, provided Tcl/Tk and (for CMPR) BLT are ported (see below). CMPR and LOGIC are developed and tested by the author primarily in UNIX (SGI and Linux computers), and increasingly with Mac OS X (10.2), but only limited testing is done in Windows (-95 through -XP). Bugs are usually fixed quickly when reported, but may not be known to the author if not reported.

Copies of the Tcl/Tk and BLT packages, are included in the Windows and Mac OS X distributions, but will need to be loaded separately on Unix platforms.

Program Documentation

The use of each program is documented in separate web pages: CMPR and LOGIC.

Acknowledgments

CMPR and LOGIC incorporate a number of FORTRAN/C programs written by others. Further, some of the programs written by me incorporate code from other people's programs, or were inspired by looking at code from others. Here are some of the sources:

• The reflection generation code traces back to the NIST AIDS*83 program (C. Hubbard, J. Stalick, A. Mighell & others),
• space group extinctions from NRC symmetry codes (A. C. Larson),
• peak fitting uses GPLSFTA (D. Cox, W. Hamilton, L. Finger & many others),
• Gaussian smoothing/peak search routines were supplied by R. Harlow and A. McGhie.
• Sections in LOGIC were inspired by the work of Ray Goehner, Mary Garbauskas, Gerry Johnson and Richard Harlow. Many subroutines used to parse and display PDF-2 (AIDS*83) entries were written by Mark Holomany.

The CMPR program provides a user interface to several auto-indexing programs that were written by other authors:

• the ITO autoindexing program was written by Jan W. Visser [J. Appl. Cryst. 2,89 (1969)], with contributions from Robin Shirley and Jerry G. Johnson, Jr.
• the DICVOL autoindexing program was written by A. Boultif and D. Louër, [J. Appl. Cryst. 24, 987-993 (1991)]
• the TREOR autoindexing program was written by Per-Erik Werner [J. Appl. Cryst. 18, 367-370 (1985)]. Note that a newer version of TREOR is due as part of the EXPO package soon, look for "Indexing by N-TREOR in EXPO" by C. Giacovazzo et al. in J. Appl. Cryst..

Mailing List

If you use CMPR, please drop me a line to get on the mailing list. (Click here.) By doing this, you will hear about updates (but not very often; nor will you get e-mail through this list about anything else). Having a list of users might help me show my management that these tools are valuable to the scientific community; this could result in more CMPR/LOGIC development.

Installation Instructions

Instructions for installing CMPR & LOGIC are covered below, with separate sections for Windows, Mac OS X and UNIX.

Windows:

For Windows, CMPR, LOGIC and other required files are most easily installed by running a single self-installing executable file.

1. If you will be using the ICDD-JCPDS PDF-2 powder diffraction database, you should install that first.

2. Download and run the CMPR/LOGIC self-installer (~7 Mb) from the NIST website: ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cmpr+logic.exe or one the CCP14 mirrors [(UK), (Canada), (US) or (Australia)].

3. Run the self-installer program, which will take you through the usual process of installing a Windows application, including offering you several choices:
   • Where should the programs be installed on your computer?

     The default answer of c:\Program files\CMPR will be fine for most people.

   • Where in the startup menu should the shortcuts to CMPR and LOGIC be placed?

     The default answer of a folder named CMPR & LOGIC will be fine for most people.

   • Should the CMPR and LOGIC shortcuts also be placed on the desktop?

     This is a matter of personal preference.

   • Should the ENCODE program be run to create the files need by LOGIC from the ICDD PDF-2 file?

     The ENCODE program reads the ICDD-JCPDS PDF-2 powder diffraction database and creates the index and pointer files needed to use the LOGIC program. If the PDF-2 file is not available, a small amount of test data is distributed with the program to use in the ENCODE program to demonstrate how LOGIC works.

   After these choices are selected, the self-installer places all files needed by CMPR & LOGIC on your disk, creates shortcuts & start-menu entries and then, if requested, starts the ENCODE script (see below) to index the ICDD database or test files to serve as an example.

Updates:
If a newer version of the cmpr+logic.exe program is available. That program may be run to update the distribution files.

On occasion, newer versions may be released only as a compressed Zip archive on the NIST web site: ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cmpr_alpha.zip or one the CCP14 mirrors [(UK), (Canada), (US) or (Australia)]. To update using this distribution, download the cmpr_alpha.zip file. Expand using pkzip, winzip or built-in software in the newer versions of Windows. Finally, merge the contents of the .zip file into the location where CMPR is already installed (for example C:\CMPR\)

UNIX:

The process of installing CMPR and LOGIC on all types of Unix systems follows the same outline:

1. If you will be using the ICDD-JCPDS PDF-2 powder diffraction database, you should install that first.

   You will either need to use a Windows PC to unpack and install the database from CDROM and then move the file to your Unix system (note that you may not access the file from two computers on a single license), or you will need to get a copy of the database from the International Centre for Diffraction Data in a format that can be read on your system (for example, a CD-ROM with the database compressed using PKZIP/WINZIP or gzip).

2. Download and install the Tcl/Tk packages

   Note that Tcl/Tk is preinstalled in many versions of Unix, such as LINUX. It is available prepackaed for installation on many other operating systems. You may need (or prefer) to compile it yourself.

3. Download and install the BLT package

   While there are some versions of BLT prepackaged, it other cases you may need to compile it yourself. Perhaps someday, it may be integrated with Tcl/Tk which would make this part of life easier.

4. Download the CMPR & LOGIC distribution.

   Obtain the CMPR/LOGIC distribution from from the NIST website: ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cmpr_alpha.tar.gz (~1.4 Mb)
   or one the CCP14 mirrors [(UK), (Canada), (US) or (Australia)].

5. Compile the CMPR & LOGIC programs.

   Select compilation options by creating files src/make.inc and logic/Make.inc (see Makefile vars below) in an editor. Then compile the programs using

        make all
   

   On Linux and SGI computers, pre-made versions of the src/make.inc and logic/Make.inc files can be loaded and the compilation performed in a single step using the command

        make linux
   

   or

        make sgi
   

   Alternately for Linux and SGI platforms, the executable files (~1 Mb) can be downloaded using from the NIST website (also available on the CCP14 mirrors):
   ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cmpr_linux_binary.tar.gz
   or ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/cmpr_sgi_binary.tar.gz

   Note that the script used by make (Makefile) use some conditional features of GNU make. If you only have access to your local system's version of make, you may need to uncomment the definition of SYSTEM and set the value of INSDIR manually.

6. Put CMPR, LOGIC and the ENCODE Tcl/Tk scripts, which are named respectively,

      .../cmpr/cmpr.tcl, 
      .../cmpr/logic/logic.tcl
      .../cmpr/logic/encode/encode.tcl
   

   into the path so they may easily be run. There are at least three ways to do this:

   • include the CMPR directories in the search path (not recommended)
   • create short shell scripts that call these Tcl/Tk scripts
   • create links in a system directory to the Tcl/Tk scripts

   This latter approach can be invoked using command:

       make install
   

   This creates links to the CMPR, LOGIC and ENCODE scripts in /usr/local/bin, /usr/bin or ~/bin (whichever is found in your path). You can force the install directory to be a different location by specifying the directory using make variable INSDIR, for example:

       make install INSDIR=/opt/bin
   

7. Index the PDF-2 database

   If you will use the ICDD-JCPDS PDF-2 powder diffraction database you will need to run the ENCODE script (see below). This script can be invoked from CMPR using the Options/Install ICDD database menu item, or by using the "pdfencode" shortcut created above, or by typing command

        wish /my/install/path/cmpr/logic/encode/encode.tcl
   

   Rerun this Encode script to upgrade the PDF-2 database.

Mac OS X:

A special binary CMPR distribution for OS X is provided, which makes CMPR easy to install. The only prerequisite is that X11 (X windows) must be installed before CMPR may be used.

1. Install X11.

   For information on loading X11, see the appropiate section in the EXPGUI documentation OSX.

2. If you will be using the ICDD-JCPDS PDF-2 powder diffraction database, you can install that either before or after installing the CMPR & LOGIC software.

   You will either need to use a Windows PC to unpack and install the database from CDROM and then move the file to your Mac (note that you may not access the file from two computers on a single license), or you will need to get a copy of the database from the International Centre for Diffraction Data in a format that can be read on your system (for example, a CD-ROM with the database compressed using PKZIP/WINZIP or gzip).

3. Download the CMPR & LOGIC programs.

   Obtain the CMPR distribution as a Mac disk image (.dmg file) from the NIST website: ftp://ftp.ncnr.nist.gov/pub/cryst/powdersuite/osx_cmpr.dmg (~7 Mb) These files can also be downloaded from the CCP14 mirrors [(UK), (Canada), (US) or (Australia)].

4. Mount the osx_cmpr.dmg file you downloaded by double-clicking on it in the Finder (this happens automatically in Safari).

   This should give you a new volume in your top-level ("Computer") finder window called "CMPRvol"; clicking on the CMPRvol icon either on your desktop or in a finder window will bring you open a finder window with a single folder ("cmpr") present. Drag the cmpr folder to a convenient location on your hard disk using the Finder.

   While you can run CMPR & LOGIC from the compressed, read-only osx_cmpr.dmg file, it runs faster when expanded into a folder on your hard disk. If you will copy the directory from the command line, be sure to use a command such as "ditto -rsrcFork" which will preserve resource fork information.

   To dismount the volume, drag the CMPRvol icon from the desktop or "computer" (top-level) finder window to the trash.

5. Using the CMPR AppleScript app.

   The OS X version of CMPR & LOGIC has an AppleScript application in the "cmpr" folder. This AppleScript will start X11, if needed and then lauch CMPR. (If you have ideas for improving the script, the code can be found in file cmpr_app.txt)

   The CMPR AppleScript can be used in two ways:

   • Double-clicking on the icon will launch CMPR so that it starts with the "file open" window in your home directory. This window can then be used to navigate to access/create experiments in other folders.

   • Dropping one or more data files will cause those files to be read. If there is more than one known file format matching the file extension, you will be given a chance to select the format. Dropping a folder onto the CMPR icon will cause CMPR to be started with in that folder. It is possible to drag both a directory and files onto the CMPR icon.

6. Installing CMPR AppleScript app in Dock or Desktop (optional).

   You may find it convenient to drag the CMPR icon to the dock, for easy access.

   Alternately, you may find it convenient to place the CMPR on the Desktop or in a folder where it will be easy to access. Note, that this CMPR app will not work correctly if copied or moved to another folder -- instead create an alias (for example using the Finder Command-L key). Then move the alias where desired.

7. Index the PDF-2 database (optional)

   If you will use the ICDD-JCPDS PDF-2 powder diffraction database you will need to run the ENCODE script (see below). The script can be invoked from within CMPR using the Options/"Install ICDD Database" menu option. It can also be accessed the "pdfencode" shortcut described above, or by typing

       /sw/bin/wish /my/install/path/cmpr/logic/encode/encode.tcl
   

   Rerun this Encode script to upgrade the PDF-2 database.

Builing CMPR on OS X, making command-line "shortcuts" to run CMPR, etc. (optional).

For people who do not like to use downloaded executables, note that is is also possible to compile and build using the Unix distribution (see above) but then you will also need to install the following in addition to X11: the g77 compiler; FINK, Tcl/Tk, BLT. Note that the build command is:

       cd .../cmpr
       make osx

If you are a "Unix enthusiast," you might be interested in setting up command line short-cut to CMPR; however, my presumption is that most people buy Macs to avoid using the command line. Those people who wish to define a command line short-cut will want easy ways to run these Tcl/Tk scripts:

     .../cmpr/cmpr.tcl, 
     .../cmpr/logic/logic.tcl
     .../cmpr/logic/encode/encode.tcl

for CMPR, LOGIC and ENCODE, respectively. If you want to run them from the command line they need to be inserted into the path so they may easily be run. There are at least three ways to do this:

• include the CMPR directories in the search path (not recommended)
• create short shell scripts that call these Tcl/Tk scripts
• create links in a system directory to the Tcl/Tk scripts

This latter approach can be invoked using command:

    sudo make install

This should creates links to the CMPR, LOGIC and ENCODE scripts in /usr/local/bin. You can force the install directory to be a different location by specifying the directory using make variable INSDIR, for example:

    sudo make install INSDIR=/opt/bin

Using the ENCODE program

Three programs must be run to read the ICDD-JCPDS PDF-2 powder diffraction database and create a series of "index" files that allow the LOGIC program to rapidly search and access records in the database. It also creates a file (Unix, ~/.icdd_files_loc or Windows, c:\icddloc.txt) that specifies the location of the database files. If this file is not present, the LOGIC program will not run and the LOGIC panel will not be available in the CMPR. Note that the LOGIC programs will look in additional locations as well, see below, but the ENCODE script writes this information in either in ~/.icdd_files_loc or c:\icddloc.txt.

To faciliate running these programs, the CMPR/LOGIC package includes a script (encode.tcl) that obtains file locations as input and then runs these programs and creates the required files. This script can be invoked within CMPR using the Options/"Install ICDD database" menu item. On Windows it can also be run as part of the self-installation program or from the Windows Start menu. On Unix, the shortcut "pdfencode" can also be created to access this script.

The menu produced by the ENCODE program appears to the right. Note that there are six input fields:

Output directory

Where files will be written; users should not need to change this.

Input PDF2 file

The location of the ICDD PDF-2 database, pdf2.dat. This may default correctly in Windows, but will need to be set on other platforms.

Local PDF2 file

The location of local PDF-2 entries, if any: It is possible to include extra entries to the LOGIC program, provided these entries are coded in the NIST AIDS*83 format. If this is done, the entries are included with the ICDD-JCPDS entries for searches and display. Please note, this capability has not been tested in many years. Contact the program author, if problems are noted.

Deleted pattern treatment: Include deleted patterns

The ICDD-JCPDS database includes entries that the editors have later flagged as "deleted" when better patterns have been obtained. Some users wish to see these entries, some do not. If these patterns are included in the index files, it is possible to eliminate them from any search later. Note that if deleted patterns are included, their entry numbers are flagged with a suffix "D".

Deleted pattern treatment: Include peak positions for deleted patterns

If deleted patterns are included, using the previous option, you have the choice to include only chemical and reference information. Alternately, peak positions may also be included for deleted patterns, in which case they will be found during peak location searches.

Write peak indexes at

Peak index files: LOGIC can be used to search for entries with peaks in particular locations and with specified intensities, referenced as a percentage of the maximum peak height. The user can select which index files are written. Two index files are always written: one containing the three strongest peaks and one with all peaks. Specify here the index files you want created, where 70 and 30 (the defaults) specify an index to peaks 70% and 30%, or greater, respectively. Any number of index files can be written.

After all input has been supplied, the "Install PDF-2" button should be pressed. This starts the three programs, which can take a few minutes to a few hours to run.

---

LOGIC looks in the following locations for database locations:

1. a file pointed to by an environmental variable, LOGIC_LOC,
2. if not successful, then file .icdd_files_loc in the user's Home directory ("C:\" in earlier versions of Windows and "C:\Documents and Settings\[user]" and in later versions of Windows.)
3. if not successful, then /usr/local/icdd_files_loc.txt
4. if not successful, then file icddloc.txt in the current working directory
5. finally, if not previously read, then C:\icddloc.txt

If the file is not found, an error message is generated in program LOGIC, but CMPR does not generate any error messages.

---

Makefile variables

The following Makefile variables may need to be set, to adapt compilation to suit your flavor of Unix.

Symbols in file src/make.inc

FC

FORTRAN compiler name (g77, f77, etc.)

FFLAGS

FORTRAN compiler command-line options

EXESUFFIX

suffix on executable files (.EXE for windows and usually blank in Unix)

Symbols in file logic/Make.inc

FC

FORTRAN compiler name (g77, f77, etc.)

CC

C compiler (gcc, cc, etc.)

F2CLIB

extra libraries needed by FORTRAN programs, if any

FFLAGS

FORTRAN compiler command-line options

CFLAGS

C compiler command-line options

LIB

name of logic subroutine library, usually ../logic.a

EXESUFFIX

suffix on executable files (.EXE for windows and usually blank in Unix)

RANLIB

set to ranlib, if needed with ar; set to echo otherwise

NAMEOPTION

Compiler flags that determine the naming conventions used in C for FORTRAN subroutines. (See below)

Options for the NAMEOPTION variable

There are differences the naming and length conventions by different FORTRAN and C compilers. To allow the code to be used on different platforms, several C macros must be defined to within a C header file (srclogic/logicmap.h and encode/logicmap.h). One of each set of values must be passed to the compiler (for example, "NAMEOPTION = -DTWOSCORE -DBIT32").

Three different methods are defined to map between FORTRAN and C names, according the value of NAMEOPTION:

ONESCORE

where one _ character is appended
The SGI f77 compilier adds an underscore (_) to the names of FORTRAN subroutines.

TWOSCORE

where one or two _ characters are appended
The GNU g77 compilier adds an underscore to the names of FORTRAN subroutines except in the case of names that already contain an underscore, g77 adds two.

NOSCORE

where no _ are appended
The HP fort77 compilier does not add any underscores.

If you see error messages of with undefined references to subroutines named ps_* or psl_* you have likely selected the wrong option here.

Three options are defined for the mapping of variable types between FORTRAN and C (for example, does INTEGER*4 correspond to long or int or even short in C).

BIT16

used for 16 bit operating systems (are there any left?)

BIT32

used for 32 bit operating systems

BIT64

used for 64 bit operating systems (none have been tested)

---

Neither the author nor the U.S. Government makes any warranty, expressed or implied, or assumes any liability or responsibility for the use of this information or the software described here. Brand names cited here are used for identification purposes and do not constitute an endorsement by NIST.

Brian Toby (Brian.Toby@NIST.GOV)
$Revision: 1.4 $ $Date: 2004/04/05 20:53:25 $