• Links to further information
• General notes
• Quick-start guide
• Getting started
• Procedure
• Miscellaneous procedure files
• Modifications log
• References

• Radermacher et al. (1987) -- description of the technique of random conical tilt.
• General information about Euler angles, and a note about Euler angles in RCT.
• Information about SPIRE, the SPIDER Reconstruction Engine, including installation.
• Download link for the Nature Protocols data set.
• A description of the reconstruction parameters.
• Multivariate data analaysis, including reference-free alignment, correspondence analysis, and classification.
• Classification and clustering tutorial.
• Documentation for verifybyview.py.
• A note about use of mirrored reference images in multireference alignment.
• Projection onto convex sets.
• Three-Dimensional Electron Microscopy of Macromolecular Assemblies, Oxford University Press, (February 2006)

• Each step in the process is bold and marked by a bullet.
• Procedure files are links to the procedure script.
• Inputs are marked in pink italics.
• Outputs created by that procedure are marked in green.
• Graphical tools are marked in red.
• Command-line instructions are written in Courier.
• Parameters will be in bold italic.
• Sequential operations are numbered. Options are lettered.
• Data extension assumed to be .spi. Adjust accordingly.
• List of inputs and outputs in description is not exhaustive.

• Before unpackaging the procedures tar archive (as described below), create a new directory, and change to that directory .
• Micrographs should be placed in the Micrographs/ directory. They needn't be in SPIDER format, but you'll then need to run convertmics.rct for the conversion.

There are two different choices for running SPIDER procedures: Using SPIRE or using SPIDER procedures directly.

1. If you are using SPIRE, then read its documentation, starting with Starting a new project. In this case, the configuration file is called rct.xml, and is located in the directory in which you unpackaged the tarball.

   Even if you are not using SPIRE, its libraries are required for various Python scripts, such as mkfilenums.py and binarytree.py so you should install it anyway.

   Installation instructions for SPIRE can be found here.

2. If you are running SPIDER at the command-line prompt, the processing steps are carried out by procedure files, which run many SPIDER operations automatically. To use a procedure in this document, if you have not unpackaged the complete tarball, then right-click the procedure filename and save the procedure to your current working directory. At the beginning of each procedure, there is a list of parameters that can be adjusted according to the particular project. Some of the procedures will call additional procedure(s). You do not need to change anything in the procedures. Use the following format to run a SPIDER procedure:
   spider rct/spi @proc
   where rct is the procedure file extension, spi is the project data file extension, and proc.rct is the procedure file.

• Prepare project directory

1. The following steps will unpackage files in the currect directory, so create a new directory if so desired.
2. Download the rct procedures tar archive
3. Unpackage the archive by typing:

   tar -xvf rct_procs.tar.gz

• Prepare parameter doc file

Many parameters are used repeatedly throughout this protocol. There are two ways to prepare the doc file: makeparams.rct or SPIRE.

1. The procedure file makeparams.rct stores these parameters. Run it interactively, and answer the questions.
2. When you start a new project in SPIRE, it will spawn a pop-up window with entries for you to fill. Some of the values will be inter-dependent, such as pixel size and maximum spatial frequency.

Either of these steps will create the document file:

¤

params:

A doc file containing the reconstruction parameters


Not all 20 parameters are used, so here is a summary of the key values:

¤	1:	zip flag -- if set to 1, micrographs will be uncompressed using gunzip.
¤	2:	format flag -- 0 corresponds to SPIDER, 3 to Z/I TIFF, etc. See complete description for additional formats
¤	5:	pixel size, in Angstroms.
¤	17:	window size, in pixels.
¤	18:	particle diameter, in pixels.

Alternatively, you can copy and modify this file from elsewhere. An example file called bak-params is present in the tarball.

• [Optional] Copy sample data

The sample data set, from Nicolas Boisset, consists of a phantom tilt pair containing worm hemoglobin. Access the data archive and unpackage it by typing:

tar -xvf data20060817.tar.gz

This command will create a directory called Micrographs/, containing phantom micrographs, and also a directory called TP1/, containing the files used to generate the micrographs.

• Generate a list of micrographs

The Python script mkfilenums.py generates a SPIDER doc file with a list of filenumbers. The syntax to make a file list of TIFF-format micrographs would be (after substituting the file pattern of your micrographs):

mkfilenums -f filenums.spi Micrographs/ribo*.tif

If SPIRE's python tools are not installed, you can generate filenums using other means, such as by manually entering the filenumbers using 'DOC CREATE' in SPIDER.

If the micrographs are stacked -- such as in the MRCS example described below -- the run this step after unstacking them.

• If the micrographs are not already in SPIDER format, then convert them

There are two general ways to convert micrographs: in SPIDER using convertmics.rct, or other software, such as XMIPP.

1. The procedure file convertmics.rct, which calls subroutine convert_p.rct, will convert, in the default example, zeiss****.tif to SPIDER format. Change the format for the input filename as necessary. Acceptable formats are specified by parameters 1 and 2 in params (described here).

2. XMIPP is also convenient for converting MRC stacks, which is written by some camera software. (Note however that MRC stacks present their own problems.) The syntax (XMIPP version 3.1) to convert an MRC stack to SPIDER would be:

   xmipp_image_convert -i mrc-stack.mrcs --swap --oroot mic:spi

   The SPIDER-format micrographs will have too many digits though (the procedures below expect four). You can either remove the extra digits, or change all of the downstream procedures to accommodate the extra digits. To perform the former, many graphical file managers allow mass-renaming. Alternatively, you can do a FOR-loop from the command line in the BASH shell. (If you're using CSH or TCSH, you can temporarily switch to bash simply by typing bash, and return to your old shell with CTRL-d or by exit).

   The syntax for removing two leading zeroes from, say, 28 files would be:
   for i in {01..28}; do mv mic0000$i.spi mic00$i.spi; done

   Now that the micrographs are unstacked, run mkfilenums.py as described above.

Outputs:


¤

Micrographs/mic****:

SPIDER-format micrographs.

• Shrink the micrographs

Here, we will choose a reduction factor such that the micrograph fits on the screen (or better yet, an even number of micrographs), and/or where the particles are visible. The procedure file shrinkmics.rct will shrink the micrographs Micrographs/mic****. Output:

¤

Micrographs/sm-mic****:

Downsampled micrographs.

HINT: The high-pass filtration step can be slow, so by setting [filter-radius]=0, you can skip this step. For micrographs, I subjectively like a high-pass filter radius of 1500 Agnstroms.

HINT: If you're too well oversampled (i.e., if your pixel size is too fine), you can save two copies of shrunken micrographs, for example, one that convieniently fits on the screen (say, a 4k image reduced by a factor of 5), and another to be used for particle-windowing (perhaps, reduced by a factor of 2). If you do the latter, remember to update your params file. The value there (in line 16) is used here only during conversion of the micrographs.

• Screen micrographs

Not all of the tilt pairs will necessarily be usable, so we will screen them. Micrographs can be screened using various display programs. To use, montagefromdoc.py, for example, cd Micrographs/ and type:

montagefromdoc ../filenums.spi sm-mic*

The tarball should have left a (hidden) file called Micrographs/.montagefromdoc with reasonable settings. The output selection docfile should be the same as the input: ../filenums. By default, two micrographs will be displayed at a time. Select them with CTRL-a or by clicking on each one.

NOTE: If the micrographs do not have the same dimensions, montagefromdoc.py will stretch them to be the same size (for display purposes only). A later step, padmics.rct, will pad micrographs as necessary.

If instead of with montagefromdoc.py), you wish to remove micrographs by editing filenums it isn't necessary to renumber the keys. The next procedure, micpair.rct, will renumber the keys internally.

• Establish pairings between micrographs

The procedure file micpair.rct reads filenums and establishes pairs of micrographs. Output:

¤

docmicpairs:

Doc file listing untilted and tilted micrographs.

This procedure file presumes that the tilted-specimen micrograph precedes the untilted-specimen micrograph. Check the output to confirm that the pairings make sense.

• Resize micrographs [Required for JWEB, not required for WEB]

JWEB has a "feature" that requires the tilt pair to have the same dimensions. The procedure file padmics.rct reads pairs as judged by docmicpairs, of micrographs Micrographs/sm-mic**** and sets them to have the same dimensions.

Outputs:


¤

Micrographs/jweb-mic{***[untilted]} Micrographs/jweb-mic{***[untilted]}_tilted:

Padded untilted-specimen and tilted-specimen micrographs, respectively.

Another "feature" of JWEB is that it expects the tilted micrograph to be second in the file list, even though it is typically collected first at the microscope. So, the tilted micrograph is named such that it appears second in the Open dialog. For bookkeeping purposes, from this point on, "micrograph number" will correspond to the number of the untilted micrograph.

• Pick tilt pairs

In this step, we need to pick the same particle in the untilted- and tilted-specimen images. There are two options for picking tilt pairs: JWEB or WEB:

1. If using JWEB:
1. From the Micrographs/ directory, open JWEB.
2. Go to File -> Open -> Image Series and select a pair of micrographs: Micrographs/jweb-mic{***[untilted]} and Micrographs/jweb-mic{***[untilted]}_tilted. Adjust the contrast as convenient.
3. Click on the Markers/Tilt Pair tab.
4. In the Marker File Number field, enter the number of the UNtilted micrograph. This number will be appended to the various file outputs.
5. Click on the Show/Edit Marker button, which will display the images.
6. Select a particle from the untilted micrograph, and the corresponding particle from the tilted micrograph. Repeat this for at least four particles, separated as far as possible in the micrograph, the farther apart the better.
   HINT: The middle mouse button will erase a pair of points if you make a mistake, or if JWEB becomes confused..
   HINT: Since the numbering is to the right of the marker, it may be easier to proceed from right to left.
7. Click on buttons Save Marker Files and Determine Theta, which will determine the magnitude of the tilt angle between the micrographs.
8. Click on the Fit Angles button. This will determine the direction of the tilt axis.
9. Click on the Draw Fitted Locations button. This will slightly adjust the locations of the fitted coordinates according to the fitting above, but does not affect the coordinates selected with the mouse.
10. Click on the Save Angles button.
11. Now, when you select a particle in the untilted image, JWEB will send the cursor to the corresponding location in the tilted image.
12. Select more particles from the tilt pair. (For the sample micrograph, there are 76 pairs. Select all of them.)
13. To save the new values, re-click the five buttons: Save Marker Files, Determine Theta, Fit Angles, Draw Fitted Locations, and Save Angles.
2. If using WEB:
1. From the menu, select COMMANDS > Tilted Particles, and select first the untilted-specimen image Micrographs/sm-mic{****[tilted]} and then the tilted-specimen image Micrographs/sm-mic{****[tilted]}.
   If in WEB you entered a number under Image reduction factor (after setting the reduction factor in OPTIONS > Image) when you run windowparticles.rct below, continue to use the parameter [shrinkFactor] that you used in shrinkmics.rct. Do NOT multiply the additional reduction factor used; using both reduction factors would be redundant.
2. In the Particle doc popup window, under Doc file number, enter the number of the UNtilted micrograph and click ACCEPT. This number will be appended to the various file outputs.
3. Select a particle from the untilted micrograph, and the corresponding particle from the tilted micrograph. Repeat this for at least four particles, separated as far as possible in the micrograph, the farther apart the better.
4. Click the middle button to open the Particle picking menu_popup.
5. Click on Determine tilt axis angles, Draw fitted locations, and Save angles in doc file.
6. Now, when you select a particle in the untilted image, WEB will send the cursor to the corresponding location in the tilted image.
7. Select more particles from the tilt pair. (For the sample micrograph, there are 76 pairs. Select all of them.)
8. To save the new values, re-click the three buttons: Determine tilt axis angles, Draw fitted locations, and Save angles in doc file.

Differences between JWEB and WEB:
• WEB does not require the pair of micrographs to be the same size, so the step padmics.rct is unnecessary.
• There is no Save Marker Files button in WEB. Coordinates are saved continuously.
• Determine Theta and Fit Angles in JWEB are now merged into one button in WEB, Determine tilt axis angles.

HINTS:
• Save the coordinates frequently. JWEB gets confused from time to time, and the main console window will state which particles have it confused. Delete those particles (using the middle mouse button), and re-pick them, as necessary.
• The particles will be numbered to the right of and slightly above the point where you clicked. So, it may be useful to start from the top right and work your way toward the bottom left.
• Your initial fit of the angles will probably be clustered in a small area (it isn't easy to find common constellations of particles in the tilted and untilted micrographs), so your clicked coordinates may deviate from the fitted coordinates as you stray from that cluster. If, while picking pairs, the fitted locations stray from the correct location, it may help to re-do the fitting.
• If you close a pair or micrographs and re-open them (with the appropriate Marker File Number), you can resume from where you left off.

• Obtain a noise image for normalization

We will be normalizing the images using the SPIDER command 'CE FIT', which fits the histogram of the background to that of a "noise" image. The procedure file makesomenoise.rct will window out a series of candidate images to use as the noise reference from a full-size micrograph Micrographs/mic**** (choose any micrograph).

Outputs:


¤	noise:	An noise reference image at random.
¤	tmpnoise/noi:	A series of candidate noise references.

You can look at the noise images tmpnoise/noi*** and copy one that looks "noisiest" to noise.spi (in the top-level directory). A second option is to copy a pre-existing noise reference from another project to noise.spi. A third option is to open as a montage tmpnoise/noi and note the image number of your preferred noise reference for the next step, windowparticles.rct.

• Window out particles from tilted and untilted micrographs.

In this step, the procedure file windowparticles.rct will window out particles from the untilted and tilted particles from the micrographs Micrographs/mic****, using the coordinates doc files Micrographs/dcu**** and Micrographs/dct****. Windowed images will be normalized using the noise file from the previous step.

For the parameter [shrinkFactor], if you picked particles from the micrographs downsampled in shrinkmics.rct enter the reduction factor used in that step. If on the other hand you loaded the full-sized micrographs (not recommended because of slowness), then enter the value used under Edit > Resize (for JWEB) or Image reduction factor (for WEB). Ultimately, this parameter needs to be the multiplicationg factor between the full-sized micrograph and the version used for particle-selection.

Outputs:


¤	listparticles:	A list of windowed particles, along with the corresponding (untilted) micrograph number.
¤	Zerodegr/stk2-winrot:	Windowed-particle stack from the untilted-specimen micrographs.
¤	Tilted/stk2-winrot:	Windowed-particle stack from the tilted-specimen micrographs.

There are additional outputs, Zerodegr/stk0-unrot and Tilted/stk0-unrot. See this note about Euler angles for information about how the different outputs relate to the 3D reconstruction.

The outputs Zerodegr/stk1-winrot_backup and Tilted/stk1-winrot_backup are generated in case high-pass filtration needs to be repeated.

• [Recommended for negatively-stained images] High-pass filter windowed images

  For negatively-stained images, it is often the case that the puddle of stain surrounding the particles dominates the classification (although maybe not the case for deep-stained specimens). In this case, it may be helpful to high-pass filter the images using a filter radius larger than the size of your particle. If you cut off too much of the low-frequency information however, the aligning power of the images will be lost. You may need to tweak the radii.

  If you are unsure whether to do these steps, feel free to skip them, but be sure to look at the reconstituted images when you run ca-pca.rct in order to see if the stain is the predominant source of variation.

  The procedure files highpass-untilted.rct and highpass-tilted.rct read stacks Zerodegr/stk1-winrot_backup and Tilted/stk1-winrot_backup, respectively, the input directories are renamed Zerodegr/Unhighpass and Tilted/Unhighpass.

  Outputs:
  

  ¤	Zerodegr/stk2-winrot:	High-pass filtered untilted-specimen image stack (overwrites previous version).
  ¤	Tilted/stk2-winrot:	High-pass filtered tilted-specimen image stack (overwrites previous version).

• Screen the windowed images for defects (micrograph edges, dust, etc.).

Here, we will screen the images for gross defects, such as micrograph edges, dust, etc. using montagefromdoc.py. Steps:

1. Open the untilted-specimen images by typing:
   montagefromdoc listparticles.spi Zerodegr/stk2-winrot.spi
   The (hidden) input .montagefromdoc should have been present in the tarball, but ensure that the output doc file is called notgood
2. Select only the bad particles. Hit PageDown and PageUp to navigate though the screenfuls of images.
3. Save the selection file with CTRL-s. A more detailed description of montagefromdoc.py can be found here.
4. If particles are to be rejected, run: removebad.rct. This procedure file, and its subroutine backup.rct, backs up and updates listparticles.
5. Screen the tilted-specimen images by typing:
   montagefromdoc listparticles.spi Tilted/stk2-winrot.spi
6. Select the bad particles.
7. If additional particles are to be rejected, re-run: removebad.rct to update listparticles.

The types of particles that should be rejected are those that will interfere with the 2D alignment or 3D reconstruction. Cut-off images, where the particle was too close to the edge of the micrograph, fall in this category. For tilted images, particles with dust should be thrown out if the dust will interfere with the reconstruction. For untilted images, dust is more tolerable; if the dust interferes with the 2D alignment, offending particles can be thrown out later. For the tilted images, this is the last opportunity to exclude them.

• Align the untilted-specimen particles using reference-free alignment.

In order to run classification, the particle will first be run through reference-free alignment. Even if there are different views of particles (orientation or conformation), the idea is that these different classes of particles will align consistently with the type of particle to which it is aligned.

There are a few different implementations of reference-free alignment, such as 'AP SR'. The procedure file used here, pairwise.rct, uses an algorithm called the "pyramidal system for prealignment construction." For more information, see the MSA documentation.

Briefly, pairwise.rct takes the first two images from the stack Zerodegr/stk2-winrot, aligns them, and averages them. In the next iteration, it will align pairs of averages, and then average them, and so forth. This type of alignment appears to be less random than does 'AP SR', which chooses seed images as alignment references.

Outputs:


¤	Zerodegr/Pairwise/stkcentavg:	Stack of centered intermediate averages, with circles overlain. See note about centering on the alignment parameters page.
¤	Zerodegr/stk3-aligned:	Aligned untilted-specimen image stack. This stack will be overwritten with subsequent alignments.
¤	Zerodegr/Pairwise/stkaligned:	Copy of aligned untilted-specimen image stack, retained locally to the Pairwise/ directory.
¤	Zerodegr/Pairwise/rfreeavg001:	Final average of all images.
¤	Zerodegr/Pairwise/docalign:	Alignment doc file.

See a description of the key parameters alignment parameters page.

• [Optional] Align average along coordinate axes

These two steps are for purely aesthetic reasons. Following reference-free alignment, the image average will be in an arbitrary orientation. These two steps attempt to reorient the average to a less arbitrary orientation.

1. The procedure file reorientavg.rct will attempt to find a bilateral symmetry axis in Zerodegr/Pairwise/rfreeavg001, and then orient that symmetry axis along the coordinate axes.

   Outputs:
   

   ¤	Zerodegr/Pairwise/malign:	Montage of reorientation candidates (see below).
   ¤	Zerodegr/Pairwise/dsolalign:	Alignment doc file for reorientaiton attempts.

   The output Zerodegr/Pairwise/malign outlines the progress of the procedure file. It takes the average Zerodegr/Pairwise/rfreeavg001 and mirrors it. The procedure file then takes the autocorrelation functions (ACF) of both images and aligns the ACFs. The best fit is rotated to vertical (solution 1) and horizontal (solution 2). The procedure file also attempts to align the images in real space (solutions 3 & 4).

2. Choose the preferred solution number from the previous step (if any), and enter it as parameter [solutionNum] in sumalign.rct. This procedure file reads the alignment information for the corresponding solution number from Zerodegr/Pairwise/dsolalign, combines it with the reference-free alignment parameters in Zerodegr/Pairwise/docpairalign, and applies them to the unaligned image stack Zerodegr/stk2-winrot.

   Outputs:
   

   ¤	Zerodegr/stk3-aligned:	Aligned untilted-specimen image stack. This stack will be overwritten with subsequent alignments.
   ¤	Zerodegr/Pairwise/sumaligned:	Aligned untilted-specimen image stack. This copy is retained locally to the Pairwise/ directory.
   ¤	Zerodegr/Pairwise/avgu001:	Realigned 2D average.
   ¤	Zerodegr/Pairwise/docalign:	Combined alignment-parameter doc file.

• Filter and shrink the aligned, untilted-specimen particles.

The procedure file filtershrink.rct low-pass filters and optionally shrinks the particle images. The next step, multivariate data analysis, will be faster if the image stack Zerodegr/stk3-aligned are made smaller. Also, the individual images will be examined subsequently and will be easier to recognize if they are low-pass filtered.
Outputs:


¤

Zerodegr/stkfiltered:

Low-pass, shrunken filtered image stack.

The default parameters are hopefully reasonable. To view the output filtered images, type:
montagefromdoc listparticles.spi Zerodegr/stkfiltered.spi

Adjust the filtration parameters as necessary, and re-run. Note the reduction factor for subsequent steps.

• Run multivariate statistical analysis (MSA) on filtered images.

The procedure file ca-pca.rct runs correspondence analysis (CA) or principal component analysis (PCA) or iterative PCA (IPCA). In principle, one can classify the image stack Zerodegr/stkfiltered without running MSA, but here we will be using it.

What this means, briefly, is that systematic variations are reduced into an arbitrary number of factors (e.g., 25 used here); in this case, the factors can be expressed as images, or "eigenimages." Each image can be reconstituted as the sum of these eigenimages, when using the proper weights. How CA and PCA differ is in the way these weights are calculated. The "importance" of each factor is the percent variation that is accounted for. The eigenimages of lower importance typically correspond to noise. Thus, an image reconstituted from the strongest eigenimages can be thought of as a type of filtered image, where some contribution of noise has been excluded.

For a more complete description of multivariate statistical analysis (or more accurately, multivariate data analysis), see Chapter 4 in Frank, 2006. For details about the specific SPIDER commands pertaining to MSA, and example outputs, see the classification and clustering tutorial.

Outputs:


¤	Zerodegr/MSA/eigenimg:	Stack of eigenimages, i.e., images showing systematic correlated variations between pixels. For more information about eigenimages (and reconstituted images), see the classification and clustering tutorial.
¤	Zerodegr/MSA/reconst:	Stack of images reconstituted using a single factor Factors can be positive or negative, hence the two versions for each. For more information about reconstituted images (and eigenimages), see the classification and clustering tutorial.
¤	Zerodegr/MSA/ps_eigenvalues.ps:	A PostScript file with a histogram of the percent variation for each factor.
¤	Zerodegr/MSA/cas_IMC:	A text file containing information about each factor and for each image For more information about the output of 'CA S', see the classification and clustering tutorial.
¤	Zerodegr/MSA/factormapAABB:	PostScript format plots for factor #AA vs. factor #BB (where BB = AA+1) Clusters are indicative of distinct classes.


Important parameters:

¤	[cas-option]:	Option of CA (option 1), PCA (2), or iterative PCA (3) One method or another sometimes crashes due to "numerical inaccuracy." I subjectively try CA first, then iterative PCA, then PCA last.
¤	[num-factors]:	Number of factors to calculate. If after you run this procedure file, all of the eigenimages look meaningful, then increase this parameter and re-run.

NOTE: For negatively-stained samples, when looking at the reconstituted images, it may be that the stain distribution is the greatest source of variation. If this is the case, go back to the high-pass filtration step above.

• Classify images.

At this step, we willl attempt to separate the images into homogeneous subclasses. There is a tradeoff between homogeneity of the subclasses, and size of the subclasses (which is related to the improvement of signal-to-noise in the class averages). In other words, if too few classes are used, dissimilar classes will be grouped together, and if too many classes are used, class averages will be likely redundant and noisy. Thus, there is some degree of subjectivity involved in the classification.

For any of the classification options presented here, choose the number of factors to use (parameter [numFactors]) based on the appearance of the eigenimages and the strength of the factors (based on the histogram), including those factors that you believe to represent true structural differences. Use of weaker factors will probably not make a appreciable difference, however.

There are three classification options given here. (For hints about the usage of the different methods, see the multivariate data analaysis page.)

To view the particles belonging to each class, refer below to the instructions about verifybyview.


Choose one of the following 3 classification options:
1. [Recommended] Hierarchical classification, using Ward's method.

1. Run hierarchical classification.

This classification method uses SPIDER command 'CL HC'. For a description of the outputs, see the classification and clustering tutorial.

The procedure file hierarchical.rct reads the MSA factor text file Zerodegr/MSA/cas_IMC.

Output:


¤

Zerodegr/Hierarchical/docdendro:

Dendrogram doc file, showing the relatedness between images. See the classification and clustering tutorial for more details.

2. Generate class averages.

In the previous step, we didn't explicitly assign classes. We simply made the dendrogram, which describes the relatedness of the images. In this step, we will explicitly pare down the dendrogram into classes.

Choose one of:
1. [Recommended] Generate binary tree

It is somewhat arbitrary where to cut off the dendrogram Zerodegr/Hierarchical/docdendro generated above. This procedure file will generate class averages for all depths until a specified cutoff, averaging the image stack Zerodegr/stkfiltered. The output class averages are arranged into the format of a binary tree, such that the average is the top-level node, the first two subclasses are at the next depth, the first two classes of each of those two classes are at the next depth, etc. While it becomes straightforward how to draw the tree, the drawback is that information is lost about the height of the branches. That is, the height of the branches in the binary tree does not direct correspond to the actual height of the branch in the dendrogram.

In the procedure file binarytree.rct (which calls averagenode.rct and update_lut.rct), the parameter [end-depth] will call for 2**[end-depth] class averages. It is better to err on the side of too high a depth and display fewer in the next step.

Outputs:


¤	Zerodegr/Tree/classavg***:	Class average for each node on the binary tree.
¤	Zerodegr/Tree/labeled***:	Class average, labeled with the class number and the number of particles.

NOTE: The format of SPIDER's dendrogram doc file changed in version 17.13. Previously the top two branches had height 1, whereas now, it's 100. If this procedure file crashes, check the parameter [top-branch].

Instructions for viewing the binary tree are given below, in the section below, on the selection of class averages.

2. Generate a pre-specified number of class averages

To use this option, run procedure file classavg.rct. This option cuts off the dendrogram dendrodoc at a specific height. The height determines the number of classes into which the dendrogram is pared. The number of classes desired is specified by the parameter [numClasses] in classavg.rct. However, because of the imprecision in how the appropriate height in the dendrogram is searched, the actual number of classes found may not exactly match the desired number of classes.

Outputs:


¤	Zerodegr/Hierarchical/classavg***:	Class averages.
¤	Zerodegr/Hierarchical/docclass***:	List of particles belonging to each class.
¤	Zerodegr/Hierarchical/listclasses:	Doc file with each information about each class Needed as an input for verifybyview.py (more information below).
¤	Zerodegr/Hierarchical/classvar***:	Class variances.

2. Run K-means classification

K-means classification uses the SPIDER command 'CL KM'. In contrast to the other classification options presented here, K-means will generate exactly the number of classes requested. The procedure file kmeans.rct reads Zerodegr/MSA/cas_IMC and, in a single step, generates K (parameter [numClasses] in the procedure file) classes from the image stack Zerodegr/stkfiltered.

Outputs:


¤	Zerodegr/Kmeans/docassign:	Doc file of analogous format to dendrograms from the other 2 classification methods.
¤	Zerodegr/Kmeans/classavg***:	Class averages.
¤	Zerodegr/Kmeans/docclass***:	List of particles belonging to each class.
¤	Zerodegr/Kmeans/listclasses:	Doc file with each information about each class Needed as an input for verifybyview.py (more information below).
¤	Zerodegr/Kmeans/classvar***:	Class variances.

3. Clustering using Diday's method

1. Run hierarchical classification.

The procedure file cluster.rct uses SPIDER command 'CL CLA', which uses Diday's method of moving centers. Like with Ward's method above, this method is hierarchical, reading Zerodegr/MSA/cas_IMC.

Outputs:


¤

Zerodegr/Cluster/docdendro:

Dendrogram doc file, showing the relatedness between images. See the classification and clustering tutorial for more details.

2. Generate class averages.

With the previous step, analogously to Ward's method above, we didn't explicitly assign classes. We simply made the dendrogram Zerodegr/Cluster/docdendro, which describes the relatedness of the images.

In the procedure file classavg.rct, we will explicitly pare down the dendrogram into classes. (I haven't been able to make Diday's method compatible with binarytree.rct, so for now, it's functionally equivalent to K-means.)

NOTE: The default parameter [class-type] in the procedure file is for Ward's hierarchical method, so change that parameter to 2.

Outputs:


¤	Zerodegr/Cluster/classavg***:	Class averages.
¤	Zerodegr/Cluster/docclass***:	List of particles belonging to each class.
¤	Zerodegr/Cluster/listclasses:	Doc file with each information about each class Needed as an input for verifybyview.py (more information below).
¤	Zerodegr/Cluster/classvar***:	Class variances.

It may be informative to see the individual particles that belong to each class. This can be accomplished by using verifybyview.py. There should be a settings file, .verifybyview, present in the tarball. For more details, refer below to the section on viewing the class averages and constituent particles.

• Select unique classes

In the next step, we will select class averages to use as references for multireference alignment. The first option applies to the output of binarytree.rct above. The second option applies to all classification options from above, including the output of binarytree.rct.

1. View and select classes from binary tree

To view the binary tree, cd Zerodegr/Tree and run:
binarytree labeled001.spi 4 goodclasses.spi

The parameter 4 on the command line above specifies the maximum depth displayed of the tree. A depth of 2, in comparison, would display only the combined average plus the two classes corresponding to the first branch in the dendrogram. Unlike the Dendrogram option in WEB, one advantage of binarytree.py is that averages are also shown for the non-terminal branches. A disadvantage is that the height of the branches in the tree are purely relative, and have no bearing to the numerical height of the branch point in the dendrogram (and thus the relatedness of the classes).

As one proceeds down the tree, the number of classes increases, and the number of particles per class decreases. As is the case with classification in general, there is a tradeoff between too big a class, such that dissimilar particles are combined, and too small a class, where the signal-to-noise ratio is low. One strategy for displaying an appropriate depth is the point at which a parent class's subclasses do not look different than the parent class.

Save the selection file with CTRL-s. Further options for the program are described on the MSA page.

The script binarytree.py requires SPIRE installation. If you don't have SPIRE installed, you can alternatively run tree.rct, which generates a (large) SPIDER image, Zerodegr/Tree/tree*. To generate a selection file of class averages, cd Zerodegr/Tree, and either categorize the montage of class averages in WEB or JWEB, or manually create the selection file interactively in SPIDER using 'DOC CREATE'. The selection file should be called Zerodegr/Tree/goodclasses.

2. View the class averages and, optionally, the constituent particles using verifybyview.py

This option can be used for any of the classification schemes described above. It can also be used in conjuction with binarytree.py to view the individual particles that belong to a class.

Open verifybyview.py by typing:
verifybyview Zerodegr/Tree   if you used the binary-tree option, or
verifybyview Zerodegr/Kmeans   if you ran K-means classification, or
verifybyview Zerodegr/Cluster   if you used Diday's clustering method, etc.

The script verifybyview.py requires SPIRE installation.


Included in the tarball is a starting .verifybyview file with mostly reasonable settings. The primary purpose of that .verifybyview file, however, was to view classes for multireference alignment, so you may wish to change the following 2 settings:

¤	Display > reverse order:	Toggling this option will display the averages starting from #1.
¤	Display > labels > text label:	Change CCC to N for the number of particles in a class.

If .verifybyview is not present (it is a hidden file, starting with a period, so to list it, type ls -a in your toplevel directory), you will need to adjust the filenames. In the initial popup window, the values should be:

¤	.spi:	Data extension. Adjust as necessary
¤	Zerodegr/Tree:	Initial directory.
¤	listclasses:	List of classes.
¤	classavg***:	Class-average template.


The first time you click on a class average, a popup menu will appear to confirm filenames, some of which are repeated from the initial popup menu. The key filenames are:

¤	docclass***.spi:	List of particles belonging to each class.
¤	Zerodegr/Filtered/flt*****.spi:	Filtered-image template.
¤	goodclasses.spi:	List of selected classes.

When you click on a class average (except for the first instance, when a popup menu appears), a montage of individual particles will appear.
NOTE: There is a limit of how many images that can be displayed in one montage. Clicking on a class with ~1000 particles will show an error message to the console window.

It is unnecessary to verify individual particles at this time. Instead, simply select the class averages that you wish to use for multireference alignment. Select the classes averages by right-clicking on them. When you are finished, save the selection by going to the File menu and selecting Save selection (or ALT-s).

• Run multireference alignment

The procedure file multirefalign.rct takes the class averages classavg*** selected in the previous step in goodclasses, re-centers them, and uses them for multireference alignment. The previously aligned image stack Zerodegr/stk3-aligned is used to generate the reference images (the class averages are derived from filtered images which may have been downsampled), and the unaligned image stack Zerodegr/stk2-winrot will be aligned to those references.

NOTE: Ensure that the classification method that you used is reflected in the procedure file label [old_class_dir]. If you used K-means, for example, use the directory Zerodegr/Kmeans. The default in the procedure file is Zerodegr/Tree.

Outputs:


¤	Zerodegr/Multiref1/stkref:	Stack of reference images used for alignment. They should resemble full-sized, unfiltered versions of the selected class averages
¤	Zerodegr/Multiref1/docalign:	Alignment parameters See the documentation for 'AP SH' for an explanation of the doc file.
¤	Zerodegr/Multiref1/selview***:	A list of particles belonging to each class.
¤	Zerodegr/Multiref1/stkcenu:	Aligned untilted-specimen image stack. This copy is retained locally to the Pairwise/ directory.
¤	Zerodegr/stk3-aligned:	Aligned untilted-specimen image stack. This stack will be overwritten with subsequent alignments.
¤	Zerodegr/Multiref1/viewavg***:	Average of particles belonging to each reference.
¤	Zerodegr/docmultiref:	Keeps track of successfully executed iterations of multireference alignment.

In addition to the same key alignment parameters as above in pairwise.rct, you may be interested in some of the consequences of [mirror-option], described in my note about mirroring references. If this parameter is set to 0 (the default), experimental images are NOT compared to mirrored references. This default is the safe choice.

• Re-filter aligned images

With newly aligned images, we will re-filter and optionally shrink the images before re-classifying them. The newly aligned image stack, Zerodegr/stk3-aligned, overwrote the old version, thus the inputs and outputs of filtershrink.rct will remain the same.

Outputs:


¤

Zerodegr/stkfiltered:

Low-pass, shrunken filtered image stack.

Adjust the parameters as necessary, although the parameters used previously should be applicable.

• Classify the particles assigned to each reference

In the next step, we will perform separate classifications for the particles belonging to each reference. In principle, the particles assigned to each reference should be identical. However, if too few references were chosen, there may be homogeneous subsets assigned to a reference. Also, if there are subtle but systematic structural differences, those difference may be noticeable upon MSA and classification. Also, junk particles (damaged particles or contaminants) may not match any references well.

The procedure file classifybyview.rct performs separate classifications on the image stack Zerodegr/stkfiltered for each of the selection doc files Zerodegr/Multiref1/selview***. As a rough measure of the quality of each particle and each class average, the cross-correlation coefficient (CCC) is computed against the reference Zerodegr/Multiref1/ref***.

Outputs:


¤	Zerodegr/Multiref1/listclasses:	List of classes for all references Also contains various statistics for each class.
¤	Zerodegr/Multiref1/classavg***:	Class averages.
¤	Zerodegr/Multiref1/docclass***:	List of particles belonging to each class.


Notable parameters include:

¤	[reduce-factor]:	Reduction factor applied in filtershrink.rct.
¤	[ca-pca]:	Option of CA (option 1), PCA (2), or iterative PCA (3) One method or another sometimes hangs due to "numerical inaccuracy." I subjectively try CA first, then iterative PCA, then PCA last.
¤	[first-view]:	If this procedure file hangs or crashes after N reference-views, change the [ca-pca] option and re-start the procedure file at N+1.
¤	[num-factors]:	Number of factors to calculate in MSA and use in classification. The classification does not appear to depend greatly on this value, but you can check the eigenimages to see if variation is not well-represented.

• Check the classification results using verifybyview

We will check the results of the classifiaction using verifybyview, which requires libraries belonging to SPIRE. See the installation instructions here.

Specifically, what we are looking for is that there is no subpopulation within each reference-view that warrants its own class. So for each reference-view, we will select the distinct class averages, and use those for the next round of multi-reference alignment and classification. If there is only one distinct class for each reference-view, you are finished with this step and can proceed to the step of verifying individual particle images.

Open verifybyview.py by typing:
verifybyview Zerodegr/Multiref1

Included in the tarball is a starting .verifybyview file with mostly reasonable settings. However, if you viewed the outputs of binarytree.rct, classavg.rct, or kmeans.rct, this file will have been modified.

The file .verifybyview is present in the rct procedures tarball archive, but if absent, you will need to adjust the filenames. In the initial popup window, the values should be:

¤	.spi:	Data extension.
¤	Zerodegr/Multiref1:	Initial directory.
¤	listclasses:	List of classes.
¤	classavg***:	Class-average template.


The first time you click on a class average, a popup menu will appear to confirm filenames, some of which are repeated from the initial popup menu. (For this popup window, the file extensions are required, but will be omitted from the list below.) The key files are:

¤	docclass***:	List of particles belonging to each class.
¤	Zerodegr/stkfiltered:	Filtered-image stack.
¤	goodclasses:	Output list of selected classes.

When you click on a class average (except for the first instance, when a popup menu appears), a montage of individual particles will appear.

It is unnecessary to verify individual particles at this time. Instead, simply select the class averages that you wish to use for the next round of multireference alignment. Select the unique classes averages by right-clicking on them. Conversely, do not select similar-looking class averages, which would then be used as separate references in the next round. When you are finished with a reference-view, save the selection by going to the File menu and selecting Save selection. If there is only one distinct class for each reference-view, proceed to the step of verifying individual particle images.

If there is an unwieldy number of classes on the main window of verifybyview.py, we can break the into subsets. Instead of the command above, type:

verifybyview Zerodegr/Multiref1/View001

...which will show only the classes corresponding to reference 1. You can proceed to the next reference by changing the directory number (e.g., View002) and pressing Update (after saving the class selection, if necessary).

• Iterate multireference alignment and classification

We will iterate multireference alignment, filtration, and classification until we obtain stable classes, i.e., no new classes appear. For a sample that doesn't have many orientations/conformations, only 2 or 3 iterations should be needed.

The specifics in the following example are for the second round of multireference alignment. The 3 steps to be iterated are:
1. Run multireference alignment

The classes selected in the previous round of multireference alignment will be used as references for the next round. For example, if you selected a 2 class averages from reference-view #1, and one class average from reference-view #2, then this next iteration will have 3 references.

The input file docmultiref keeps track of successfully executed iterations of multireference alignment, and is updated. The default parameter [multirefIterNum] of -1 will use the most recent multireference alignment as input. If you do not wish to use the last multireference alignment, override this parameter with the iteration number that you wish to use.

2. Filter the aligned particles

Run filtershrink.rct as previously.

3. Classify the particles assigned to each reference

Run classifybyview.rct, set the parameter [multirefIterNum] if you do not wish to operate on the last iteration of multireference alignment. This parameter will be present in many steps from this point.

4. Check the classification results using verifybyview

Select unique classes using verifybyview.py To open it, type:
verifybyview Zerodegr/Multiref2/View001


Remember to update number of the directory, i.e., which iteration of multireference alignment you ran.

Iterate these 4 steps -- multirefalign-descend.rct, filtershrink.rct, classifybyview.rct, and verifybyview -- until no new classes appear.

• Verify individual particles from last iteration of multireference alignment

When we compute the 3D reconstruction, we will use only the highest-quality particles. To select these particles, we will use verifybyview.py. To start it, assuming you performed 2 rounds of multireference alignment, type:
verifybyview Zerodegr/Multiref2/View001

You should have screened the class averages in the previous step, so the filenames should be appropriate. In fact, if the last directory where you viewed class averages was one of the Zerodegr/Multiref2/View### directories, you do not need to include the directory name on the command line either.

Now, when you open a montage of individual particle images, select the images that you want to keep using the left mouse button. Conversely, exclude the particles that don't resemble the reference image. To crudely assist selection of the images, the particles are sorted from highest CCC to worst. A more thorough description of verifybyview.py can be found here.

Outputs:


¤	Zerodegr/Multiref2/View###/goodclasses:	List of classes with selected particles for each reference-view.
¤	Zerodegr/Multiref2/View###/byhand***:	List of selected particles for each class.

NOTE: When you finish going through the class averages for a particular reference-view, be sure to save the list of good classes in addition to the list of good particles. The good-class list can be saved by going to the File menu and selecting Save selection, or by clicking on the Save+Update button, or with the shortcut Alt-S

• Combine selected particles from selected classes

Following the previous step, the selected particles are distributed over multiple docfiles byhand***, listed in goodclasses from multiple reference-views Zerodegr/Multiref2/View### (once again assuming two iterations of multireference alignment + classification). In this procedure file, combinegoodclasses.rct, we will combine the byhand*** files into one doc file for each reference-view.

Output:


¤

Zerodegr/Multiref2/goodsel***:

List of selected particles in each reference-view.

If in your final iteration of multireference alignment, you set parameter [mirror-option] is set to 1, there will be additional outputs for the lists of flipped and un-flipped particle images, called goodsel***-flip and goodsel***-noflip, respectively, in the Zerodegr/Multiref2 directory. See the note about mirroring references for more information.

• Average selected images

In this step, we will generate averages of the good particle images. These new averages should look cleaner than those from the multireference alignment, which also included bad particles. (Because of model bias, however, the bad particles may not have degraded the old average too much.)

The procedure file viewaverages.rct and its subroutine viewaverage1.rct read the aligned-image stack Zerodegr/stk3-aligned and averages the images according to the good-particle selection files Zerodegr/Multiref2/goodsel*** (assuming again that you performed 2 rounds of multi-reference alignment).

Outputs:


¤	Zerodegr/Multiref2/stkgoodavg:	Stack of average images for each view.
¤	Zerodegr/Multiref2/stkgoodvar:	Stack of variance images for each view.

If in your final iteration of multireference alignment, you set parameter [mirror-option] is set to 1, there will be separate outputs for the flipped and un-flipped particle images, ending in -flip and -noflip, respectively, in the Zerodegr/Multiref2 directory. See the note about mirroring references for more information.

• Select well-populated classes

Following particle-verification, not all of the classes will have an adequate number of particles for a 3D reconstruction. (This exact number will depend on the size, symmetry, etc., of the particle, and on the specimen-preparation technique, i.e., negative stain or cryo.) We will select only the classes with enough particles. Type:
cd Zerodegr/Multiref2/ ; montagefromdoc sorted-viewaverage.spi stkgoodavg.spi
(assuming you performed 2 rounds of multireference alignment)

Set the output filename to goodviews. To show the number of particles in each average, set doc file column for label to 2. Select the classes for which you wish to compute a 3D reconstruction, and save that selection (e.g., using CTRL-s).

• Center tilted-specimen particles

The goal of any 3D reconstruction technique is, for each 2D image, to obtain 2 translational parameters, and 3 orientational parameters. By this stage, we have all 3 orientation parameters. (One is the magnitude of the tilt, the second is the direction of the tilt axis with respect to the coordinate axes, and the third is the in-plane rotation angle of the untilted-specimen images.) The 2D alignment of the untilted-specimen images cannot be applied to the tilted-specimen images, so we will need to translationally align the tilted-specimen images separately.

The procedure file centertilt.rct uses a reference-free alignment to align the images in stack Tilted/stk2-winrot translationally but not rotationally. It is adapted from pairwise.rct, which was used above.

Outputs:


¤	Tilted/Pairwise/stkcentavg:	Stack of centered intermediate averages. See note about centering below.
¤	Tilted/stk3-cent:	Stack of translationally aligned tilted-specimen images.
¤	Tilted/Pairwise/rfreeavg001:	Final average of all images. Note that the images are not rotationally aligned.
¤	Tilted/docpairalign:	Alignment doc file.

NOTE: The parameter [cg-option], specifies how the center of gravity for the intermediate average will be computed, if at all. The options are:

1. No centering performed
2. Centering performed using 'CG PH'
3. Centering performed by rotating the image 180 degrees and aligning to itself

All three options are written to Zerodegr/Pairwise/stkcentavg If one is significantly better than the one you used, re-run this with the better option.

If this procedure file runs slowly, you can save time by centering only the selected particles. Change the parameter [verified-yn] to 1. This will change the input from listparticles to Zerodegr/Multiref2/combinedgood (assuming once again that you ran 2 iterations of multireference alignment).

To look at the centered tilted particles, type:
montagefromdoc listparticles.spi Tilted/stk3-cent.spi

• Store Euler angles

In this step, the procedure file storeangles.rct collects information about the 3 Euler angles from the micrograph-parameter doc files Micrographs/dcb**** and the last multireference alignment doc file Zerodegr/Multiref#/docalign This Euler-angle doc file will be used in the 3D-reconstruction step.

Outputs:


¤	Tilted/dangles:	Euler-angle doc file to be used for the 3D reconstruction.
¤	Tilted/dang-unrot:	More informative Euler-angle doc file for instructional purposes. See note about Euler angles.

Changing parameter [mirror-yn] will reverse the handedness of the structure. Whether this flag should be set to 0 or 1 depends on numerous variables in the data collection and processing, such as the direction in which the goniometer is rotated, how the micrographs are digitized, etc. Unfortunately, it is difficult to determine from first principles whether the handedness needs to be flipped, so the safest way to know how to set this parameter is to generate a RCT reconstruction of a sample with known handedness. Once the data-collection & image-processing conventions are set, change them as seldom as possible, or else you may need to re-test for flips in handedness.

• [Optional] Generate symmetry files

This step is primarily for the Nature Protocols RCT data set, but can be tailored for other symmetric samples. The procedure file d6symmetry.rct will generate files that contain the Euler angles for equivalent positions in each asymmetric unit.

The worm hemoglobin used in the Nature Protocols data set has D6 symmetry, and has two main types of views: top views and side views. There were also 4 particles that were neither top views nor side views, but they will not be used for the reconstruction. Also, there were two types of side views in that data set, but both can use the identical symmetry file since the symmetry axis is in the same direction in both.

Outputs:


¤	d6top:	Symmetry doc file where the 6-fold axis is along z.
¤	d6side:	Symmetry doc file where the 6-fold axis is in the xy plane.

To generate a doc file corresponding to a different symmetry, use the command 'SY DOC'.

• Compute 3D reconstructions for each class

The procedure file bpclass.rct and its subroutine bp3f.rct will compute 3D reconstructions for each class, whose particles are listed in each Zerodegr/Multiref#/goodsel***, The aligned tilted-specimen images Tilted/Aligned/cent***** will be reconstructed using the Euler angles found in Tilted/dangles. An optional symmetry file can be given, but in the absence of symmetry the default * is given.

Outputs:


¤

Volumes/vcla***:

Reconstructions for each class.

For the Nature Protocols RCT data set, instead run bpclass.rct. Make sure that the parameters [sideViewClassA] and [sideViewClassB] (by default in the procedure file, classes 1 & 2) correspond to side views, to which the side-view symmetry file Volumes/d6side will be applied. Similarly, make sure that the parameter [topViewClass] corresponds to the top view, (by default, class 4) to which the top-view symmetry file Volumes/d6top will be applied.

• Filter 3D reconstructions

The procedure file volfilt.rct Fourier filters one or all reconstructions Volumes/vcla***.

Outputs:


¤	Volumes/vflt***:	Filtered reconstructions.
¤	Volumes/docvolfilt:	Editable doc file containing filter parameters. See note below.


Parameters:

¤	[single-class]:	Class number whose reconstruction will be filtered. The default value of 0 will filter all class volumes. To filter a single volume, perhaps to fine-tune a specific class, enter the number of that class
¤	[header-type] [header-param1] [header-param2]:	Filter parameters. The defaults, respectively, are 7 (for Butterworth low-pass) and 37.6 & 28.2 f or pass-band and stop-band (in Angstroms)
¤	[override-yn]:	If pre-existing doc file Volumes/docvolfilt exists, setting this parameter to 1 will override the filter parameters in the procedure-file header and will use the values in the docfile instead. More below.

The reason for the usage of the file Volumes/docvolfilt is that you may want to fine-tune the filter parameters for a single class volume without having to filter all of them. Also, it may be the case that different class volumes warrant different filter parameters (if, for example, the number of particles varies). This doc file can serve as a record of what filter parameters were used.

• [Optional] Run projection onto convex sets (POCS)

  Detailed information about POCS can be found here. Briefly, POCS is an iterative way to try to minimize the effects of the missing cone.
  The steps involved in POCS are:

  1. Generate a binary mask
  2. [Optional] Choose a projection by which to gauge the progress of POCS
  3. Run POCS

• [If multiple classes] Align volumes to each other

  If you have multiple classes, it remains to be seen whether the classes correspond to the same structure. You should have a rough idea after playing with the volumes in Chimera. The procedure file volalignprj.rct and its subroutine montageprj.rct attempt to find a rigorous alignment between two volumes Volumes/vflt*** (or Volumes/volpocs***-filt).

  Note that this procedure file aligns two volumes to each other. Choose one volume as an alignment reference, to which the rest of class volumes will be aligned. After an initial alignment using a coarse search grid using 'PJ 3Q', a finer search will be performed using 'OR 3Q' (which needs an initial estimate for optimal function).

  Parameters:

  ¤	[exp-num]:	Class number for the volume to be aligned.
  ¤	[ref-num]:	Class number for the reference volume.
  ¤	[num-solns]:	Number of solutions that you wish to compare (as 2D projections) Note that multiple solutions may converge to the same refined solution.
  ¤	[dtheta]:	Angular increment for coarse search. A value of 15 degrees corresponds to 83 projection directions.
  ¤	[shift-range], [shift-step], [first-ring]:	2D search parameters as described in 'AP SH'.

  
  Outputs:
  

  ¤	Volumes/Alignment/dalv***:	Doc file with initial and refined alignment parameters.
  ¤	Volumes/Alignment/vali***:	Aligned class volume. Only written for the best match, but see note below.
  ¤	Volumes/Alignment/mproj:	Stack file of projections for each projection direction, and for the best [num-solns] solutions. More below.
  ¤	Volumes/Alignment/prj_***:	Reprojections of the reference and experimental volumes.
  ¤	Volumes/Alignment/list2merge:	Editable list of aligned class volumes with preferred alignment solution. More below.

  The first row of each slice in mproj*** shows the reference projection. Each additional row corresponds to one alignment solution, from "best" to "worst." The left-hand image is the re-projection following the initial, coarse alignment, and the right-hand image is the re-projection following the refined alignment using 'OR 3Q'.

  If the better alignment solution is not solution #1, then edit in Volumes/Alignment/list2merge column #2, labelled SOLUTIONNUM. If you do wish exclude the aligned class volume in the merging step altogether, set this value to 0 in Volumes/Alignment/list2merge.

  The aligned class volume Volumes/Alignment/vali*** is written only for the bit fit as judged by CCC, but the lower-ranked solutions can be used when merging the class volumes as described above by editing the Volumes/Alignment/list2merge file. A class volume using your preferred volume-alignment parameters can be optionally written in the next step, mergevols.rct, by setting the parameter in that procedure file [classvols-yn] is set to 1.

  The re-projections prj_*** will illustrate the missing-cone artifact better than isosurface presentations.

• [Optional] Merged particles from aligned class volumes into one reconstruction

You will need to decide subjectively whether the aligned class volumes are sufficiently similar to merge. The procedure file mergevols.rct combines data from many steps.

Inputs:


¤	Tilted/stk3-cent:	Aligned-image stack.
¤	Zerodegr/Multiref#/goodsel***:	Good-particle selection files.
¤	Tilted/dangles:	Euler-angle doc file.
¤	Volumes/Alignment/dalv***:	Volume-alignment doc files.
¤	Volumes/Alignment/list2merge:	List of classes to merge.


Outputs:


¤	Tilted/dang-merged:	Euler angles taking into account the volume alignment.
¤	Zerodegr/Multiref#/mergedsel:	Selection file containing particle numbers for merged classes.
¤	Volumes/vtot999:	Merged reconstruction.
¤	Volumes/vtot999-flt:	Filtered merged reconstruction.

If the parameter [classvols-yn] is set to 1, volumes for each class will be reconstructed of the form Volumes/vtot***, using the merged Euler angles. They should look very similar to the vali*** reconstructions from the previous step, volalignprj.rct, although in this case the volume will have undergone one fewer interpolation. Also, if you did use the best-fit solution in volalignprj.rct, then this aligned class volume will be the only version.

Filter parameters are used like in volfilt.rct above.

• Prepare files for reference-based alignment

Now that you have a low-resolution structure to used a reference, you can use model-based alignment to perform subsequent alignments.

Before running the next procedure file, download the ( reference-based alignment tarball and unpackage by typing:
tar -zxvf spiproject.tar.gz

The tarball will create a directory called myproject/
For the example here, it will be assumed that the tarball was extracted while in the toplevel RCT directory, in which case myproject/ is a subdirectory. Feel free to change the name, but remember to change it in the ensuing procedure file.

The procedure file prepare-prjmatch.rct converts the files from the RCT protocol into a form that can be used in the reference-based alignment protocol.

Inputs:


¤	params:	Parameter doc file.
¤	Volumes/vtot999-flt :	Filtered, merged volume If you didn't merge classes, use a Volumes/vflt*** class volume.
¤	docmicpairs:	Micrograph-pairing doc file.
¤	listparticles:	List of particles. See note below.
¤	order_picked:	micrograph-to-particle lookup-table.
¤	Micrographs/dct****:	Tilted-specimen coordinates doc file.
¤	Tilted/stk2-winrot:	Tilted-specimen particle image stack. See note below.

The total list of particles listparticles will be used instead of only the particles that were included in the reconstruction in case the previously-excluded particles can have their orientations determined now that there is a model.

Only the tilted particles will be used in the projection-matching step. With the untilted particles, most likely they would be overrepresented, which would cause problems in the reconstruction. Also, instead of Tilted/stk2-winrot, you could opt to use Tilted/stk0-unrot, which have undergone one fewer interpolation. However, if you needed to high-pass filter the stk2-winrot images, you will need to go back and filter the stk0-unrot images similarly.

Outputs (assumed to be in myproject/):


¤	params:	Parameter doc file (identical to input).
¤	reference_volume:	Reference volume.
¤	Power_Spectra/sel_group:	Dummy list of defocus groups. See note about CTF-correction below.
¤	sel_micrograph:	List of micrographs.
¤	Power_Spectra/def_avg:	Dummy list of defocus values. See note about CTF-correction below.
¤	Particles/win/winser_****:	Particle stacks, for each micrograph.
¤	Particles/good/ngood****:	Particle selection doc file, for each micrograph. If you plan to re-verify the particles, instead call these files Particles/win/sel_particle_****.
¤	Particles/coords/sndc****:	Coordinate doc files.

As of yet, CTF-correction is not implemented in this RCT implementation. A few dummy files related to defocus and defocus groups will be written by prepare-prjmatch.rct anyway. (Set your expectations for the resulting reconstruction accordingly.) When you reach a step where there is an option to account for the CTF or not, be sure to use the option that disregards the CTF, namely apsh.spi instead of apshgrp.spi.

You should now have the files in a form that can be understood by the reference-based alignment protocol. Begin with the section entitled "Alignment" with the step sel_by_group.spi. If you are going to re-verify the particles (in which case you should have named the output particle lists Particles/win/sel_particle_**** instead of Particles/good/ngood****), then you should begin in the section entitled " Particle Picking and Selection" with either the (optional) step pfilt.spi or the next step, entitled "Verify the automatically selected particles". Alternatively, if you are going to use classification-based particle verification, then begin in the section entitled " Particle-picking" with the step numberparticles.spi.

Good luck!

• Projection onto convex sets

• 2015-08-13 -- refproj.rct shows power spectra of re-projections, to illustrate missing cone
• 2015-08-10 -- classes from multirefalign.rct and classifybyview.rct can be viewed (and selected) all at the same time
• 2015-08-06 -- centering of tilted images now uses pyramidal scheme
• 2015-08-05 -- pairwise.rct and centertilt.rct draw circles on averages corresponding to inner and outer alignment radii
• 2015-08-05 -- ca-pca.rct and classifybyview.rct get image dimension automatically
• 2015-07-29 -- windowparticles.rct allows choice of selected noise image from makesomenoise.rct
• 2015-07-16 -- separated Nature Protocols reconstruction procedure, bpwormhb.rct, from bpclass.rct
• 2015-07-16 -- number of CPUs is now user-defined
• 2015-07-14 -- added procedure and Python script to covert from EMAN2's e2RCTboxer.py (under construction)
• 2015-03-17 -- bug fix in volalignprj/montageprj
• 2014-03-06 -- using stacks for the windowed particles
• 2014-03-06 -- changed assumed extension to '.spi'.
  (Conversion/display programs sometimes expect this extension for SPIDER files.)
• 2014-03-04 -- automatically looks for last iteration of multireference alignment
• 2014-02-04 -- pairwise.spi -- using named registers, error-checking for alignment radius
• 2013-12-12 -- tar archive, cosmetic changes
• 2012-12-20 -- posted to local Techniques directory
• 2012-12-20 -- made SPIRE configuration file

1. Frank J. (2006) Three-Dimensional Electron Microscopy of Macromolecular Assemblies. (Oxford University Press, New York, NY).
2. Marco S, Chagoyen M, de la Fraga LG, Carazo JM, Carrascosa JL. (1996) A variant to the random approximation of the reference-free algorithm. Ultramicroscopy 66: 5-10.
3. Radermacher M, Wagenknecht T, Verschoor A, Frank J. (1987) Three-dimensional reconstruction from a single-exposure, random conical tilt series applied to the 50S ribosomal subunit of Escherichia coli. J Microsc. 146:113-36.
4. Shaikh TR, LeBarron JS, Trujillo R, Baxter WT, and Frank J. (2008a) Particle-verification for single-particle reconstruction using multivariate data analysis and classification. J Struct Biol 161: 41-48.
5. Shaikh TR, Gao H, Baxter WT, Asturias F, Boisset N, Leith A, and Frank J. (2008b) SPIDER image-processing for single-particle reconstruction of biological macromolecules from electron micrographs. Nature Protocols 3: 1941-74.
6. Huang T, Shaikh TR, Gupta K, Contreras-Martin LM, Grassucci RA, Van Duyne GD, Frank J, Belfort M. (2011) The group II intron ribonucleoprotein precursor is a large, loosely packed structure. Nucleic Acids Res. 39: 2845-54.