Three Dimensional Reconstruction of Single Particle Specimens using Reference Projections With Defocus Groups

Introduction

This page describes methods for creating a 3D reconstruction from electron micrographs using defocus groups. Once a set of particle images has been obtained, an initial 3D reconstruction is calculated using coarse projection angles. This is followed by refinement, which iteratively adjusts the angles for finer resolution. The CTF is accounted for by computing reconstructions of groups of particles from micrographs with simlar defocus values ("defocus groups"), and CTF-correction is performed when these reconstructions are merged.

An alternative method is available for creating a 3D reconstruction from electron micrographs without defocus groups. The advantages of using defocus groups is as follows. First is that it can readily account for the non-uniform distribution of signal-to-noise in projection data (Penczek, 2012). Second, we find that reconstructions using particle-level CTF-correction sometimes show artifacts when using iterative backprojection methods, such as 'BP RP' or 'BP CG', whereas the use of defocus groups does not present such limitations. However most laboratories have abandoned the use of defocus groups.

In comparison, CTF-correction is applied at the level of windowed particle images offers its own advantages. First, it circumvents one of the approximations when using defocus groups, namely that all particle in a defocus group follow the same CTF profile. At high resolution, where the CTF oscillates more rapidly, this assumption will not hold. Second, parallelization can be more efficient, since groups can be of identical size, independent of the number of particles at each defocus. Third, particles from what would be sparsely populated defocus groups need not be thrown out. Lastly, interoperability with other software packages which correct for the CTF at the level of particle images will be more straightforward.

The procedures are described in greater detail below.

Contrast Transfer Function (CTF) Estimation

The CTF is estimated for each micrograph, and the micrographs are sorted into defocus groups.

Particle Picking and Selection

Identifies likely particles from the micrographs, and extracts windowed images of individual particles from the micrographs.

Alignment

The particle images are aligned to reference projections using shifts and rotations.

Averaging

The averages of each reference view are computed, to assess the distribution of projection views.

3D Reconstruction

An initial volume is calculated by back projection, along with an estimate of its resolution.

Refinement

Initial alignments are iteratively improved, as each projection is given a chance "to find a better home" in terms of its orientation and phase origin, thus improving the resolution of the initial reconstruction.

Additional methods

Other useful methods, such as classification, difference maps, and frequency correction by X-ray scattering. Additional details about these methods may be found at the techniques page.

Links to further information

• Glossary.
• Reconstruction strategies.
• Flowchart of operations.
• Classification-based particle verification.
• Format information for doc files.
• Description of reconstruction parameters.
• Using images from a ZI scanner.
• Handling circular images from Quantifoil grids.
• References.

In the sections below, various file types are denoted by different fonts:

• Each step in the process is bold and marked by a bullet
  procedure files are links to the procedure script,
  input files for that procedure are marked in pink,
  output files created by that procedure are marked in green.,
  Sometimes more than one approach is available. Lists starting with lowercase letters indicate that you should choose
  1. one approach, or
  2. the other approach,
  but not both (unless you want to compare the two techniques)
  

  If you have installed them, SPIDER's Python Tools (marked in brown) are usefull for analyzing some results.

There are two different choices for running SPIDER procedures: Using SPIRE or using SPIDERprocedures directly.

1. If you are using SPIRE, then read how to run SPIDER procedure files in SPIRE. SPIRE will create the necessary top level directory.
2. If you are running SPIDER at the Unix prompt the processing steps are carried out by procedure files, which run many SPIDER operations automatically. To use a procedure in this document, click the procedure filename and copy the procedure to your current working directory. At the beginning of each procedure, there is a list of parameters that can be adjusted according to the particular project. Some of the procedures will call additional procedure(s) (listed below the procedure filename). You need not change anything in the procedures. Use the following format to run a SPIDER procedure:
   spider spi/dat @proc
   where spi is the procedure file extension, dat is the project data file extension, and proc.spi is the procedure file.

The data below are sometimes shown plotted with gnuplot. See the Gnuplot manual

After each SPIRE step or after running each procedure file, check the output files to make sure the results are sensible. If you are not sure what is "sensible", ask an expert.

• Create a project directory containing the required subdirectories and procedure files

1. If using SPIRE it will create the necessary top level directory and populate the directories with the current set of procedure files.
2. If using SPIDER procedures, the procedure files below are packed into the zipped tar file spiproject.tar.gz Copy this archive to your working directory and unpack it:
   tar -zxvf spiproject.tar
   This will create a directory called myproject, containing the procedure files and subdirectories for a reconstruction project. Rename myproject to the name of your project directory, usually the same three letters as the data extension, e.g.:
   mv myproject acn

3. If using data from Nature Protocols for testing, you can untar the data set and use the directory structure as your project directory.

   The data from the tar archive will then be already in place. However you must replace the procedures from the data set and stack the particle images before using the set of current procedures. The procedure nat2stk.spi reads win/ser******, ../sel_micrograph, good/ngood***, and order_select to stack images by defocus groups and creates:
   

   ¤	Alignment/data***:	Stacks of particle images.
   ¤	Alignment/sel_particles_***:	Lists of particle images by group.
   ¤	Alignment/sel_group:	Group selection doc file.

   It also renames the data set's micrograph files as necessary.

• Create a parameter document file

  Some information is stored in project-wide document files that are used by many procedure files. Run the interactive procedure makeparams.spi in the top directory of your project. It makes a document file:
  

  ¤

  params:

  A doc. file containing the reconstruction parameters

[Optional] Resize reference volume

• A reference volume is required. If the pixel size of your reference volume (key 5 in the params file) and/or the dimensions do not match the dimensions of the data that you will be processing, then you must run the procedure resizevol.spi in the top-level project directory to resize the volume. This creates:
  

  ¤

  reference_volume:

  Resized reference volume.

  The reported interpolated pixel size may not exactly match the target pixel size due to rounding errors, but the values should be close.

• Place micrograph files in Micrographs directory

  Files of the digitized micrographs should be placed in the Micrographs directory if they are not already there. For example:
  mv /actual/location/of/micrographs/raw*   Micrographs/

  The procedures below can accept SPIDER, Hiscan TIFF, PerkinElmer, ZI, and Nikon Coolscan outputs. Any of these files may have been compressed with gzip. Because micrographs require so much disk space, it may be desirable to keep the Micrographs in another location, and make Micrographs a symbolic link. In that case, cd to your project directory, remove Micrographs, and make a symbolic link:
  rmdir Micrographs
ln -s /the/actual/location/of/the/micrographs   Micrographs

• Create a selection doc file for micrograph file numbers

  The procedure makefilelist.spi creates a micrograph selection doc file containing non-consecutive file numbers in the top-level project directory:
  

  ¤

  sel_micrograph:

  Micrograph selection doc file listing the numbers of the micrograph files to be processed.

  The text part of the filenames are entered as templates in the top level procedure files - see below). If you need to add/exclude files from this list, use a text editor to edit the appropriate lines, then use the 'DOC REN' operation in SPIDER to renumber lines in the selection doc file.

For more information, see;

• Contrast transfer function correction tutorial.
• ctfdemo, a program that demonstrates the effects of various parameters on the CTF.
• Penczek et al. 1997 for a fuller discussion.

• Calculate the power spectrum for each micrograph and determine defocus using one of the following three methods

  1. Use SPIDER procedure: powdefocus.spi which calls SPIDER procedure loadmic.spi to read each micrograph, computes an averaged power spectrum, and estimates defocus using SPIDER operation: 'CTF ED'. This creates four output files for each micrograph:

     ¤	power/pw_avg****	2D power spectra, each is an average of many smaller power spectra calculated over the surface of the micrograph.
     ¤	power/roo****	Doc files of 1D profiles of the rotationally averaged power spectra. These doc files can be plotted with gnuplot or pyplot
     ¤	power/ctf****:	Doc files of the background-subtracted 1D power spectra.
     ¤	defocus:	A doc file of defocus and astigmatism values.

     The ctf**** files can be verified with ctfmatch.py
  2. Use SPIDER procedure: ctffind.spi to read each micrograph using loadmic.spi, compute an averaged power spectrum, and estimate defocus, astigmatism, and cut-off frequency using SPIDER operation 'CTF FIND'. This invokes the MRC ctffind3 program and creates two output files for each micrograph:

     ¤	power/powchk***	2D power spectra, each is an average of many smaller power spectra calculated over the surface of the micrograph and half is replaced by an idealized power spectrum.
     ¤	defocus:	A doc file of defocus and astigmatism values.

  3. Run powdefocus.spi as in a. to create power spectra: power/roo***. Then use CTF from doc file in WEB to read power/roo*** and manually fit a CTF model to the 1D profiles.

• Check the rings in the power spectrum images.

  Make a montage using WEB (reduction factor 4) to look at these power spectra. They are equivalent to diffraction patterns obtained in the optical diffractometer. It is absolutely important that you screen these spectra before proceeding any further. Things to look out for:

  • Are the Thon rings round?
  • Do the rings extend to high resolution?
  
  See this example:

  

  You can discard micrographs that show one of the following artifacts, by removing them from the sel_micrograph micrograph selection doc file:
  • Rings that are cut off unidirectionally (evidence of drift).
  • Rings that are elliptical, or even hyperbolic (evidence of astigmatism).
  See this example:

  

• Assign defocus groups according to the power spectrum.

  defsort.spi: reads a defocus doc file generated by one of the above methods, and the sel-micrograph micrograph selection file and assigns the micrographs to a tentative defocus group. It creates:
  

  ¤

  def_sort:

  A doc file with the group assignments sorted by defocus.

  These automated defocus group assignments are only a starting point, however. You must manually inspect the defocus groupings to see which micrographs belong together, ensuring that they are "in phase".

  Spectral profiles (that is, the roo*** or ctf*** files) from the same defocus group should be plotted together, to ensure that their zeroes (minima) line up, and do not cancel out. Plot the power spectrum for each micrograph and assign them to defocus groups such that the curves are in phase. Use either:

  1. ctfgroup.py
  2. pyplot.py
  3. gnuplot
     Gnuplot commands can be saved in a text file, such as plotpower.plot, then run in gnuplot using the load command:
     gnuplot> load 'plotpower.plot'

     

  When you have determined a satisfactory set of groupings, update the docfile: def_sort, which is used in the next step.

• Compute averages for each defocus group

  defavg.spi, uses the updated values in def_sort, to compute the averages for each defocus group, creating:
  

  ¤	def_avg:	A micrograph selection doc file containg average defocus level, and defocus group assignment.
  ¤	sel_group:	A defocus group selection doc file containing average defocus level.

• Select a background noise file

  A noise image is required to normalize the backgrounds of the particle images. Do one of the following:
  1. Use a noise image from a previous project (this is the easier way). The window sizes do not need to match.
  2. Use the procedure: noise.spi (which calls convert_p.spi) to read the micrograph images: ../Micrographs/raw**** and create:
     

     ¤

     noise:

     A noise image.

     This noise file is randomly selected from a set of files (tmpnoise/noi***) that are windowed from regions in the micrograph that appear not to contain any particles. View the selected noise file. If it has any structure (i.e. particles or junk), then select another one of the candidates and copy it to: noise.ext.

• Run automatic phase of particle selection.

  Either of the following two approaches can be used. They both require the image format conversion procedure: convert_p.spi
  1. SPIDER procedure pick.spi (which calls: convert_p.spi and pick_p.spi) applys correlation with a Gaussian blob to window particles.
     The micrographs are decimated by 1/4, Fourier filtered with a Gaussian low pass filter, and searched for peaks over regions corresponding to particle dimensions. Particles are windowed out and centered, then peak locations are corrected, and particles are windowed again. This procedure uses histogram equalization. Output files:
     

     ¤	win/winser****:	Stacked particle images.
     ¤	win/sel_particle****:	Particle selection doc file for each micrograph.
     ¤	coords/sndc****:	Peak coordinates for particles.

  2. SPIDER procedure lfc_pick.spi (which calls convert_p.spi and pickparticle.spi) uses the Local Fast Correlation algorithm to window particles. This procedure uses local cross-correlation calculated using Fourier methods outlined by Roseman (2003). This implementation is described in Rath and Frank (2004). A reference volume is required to generate projections at desired Eulerian angles for searching different orientations of the search image in the micrograph. Procedure lfc_pick.spi also performs a histogram-fitting normalization of the images. Output files:
     

     ¤	win/winser_****:	Stacks of particle images for each micrograph.
     ¤	win/sel_particle_****:	Particle selection doc file for each micrograph.
     ¤	coords/sndc****:	Pixel coordinates for center of particle images.

• [Optional] Low-pass filter particle images before particle selection

  The procedure pfilt.spi filters a set of images, adjusting filter settings for each micrograph, based on its defocus and creates:
  

  ¤

  flt/winser_***:

  Stacks of low-pass filtered particle images.

  Manual selection of good particles (the next step) is easier if the particles have been low-pass filtered beforehand. Use the filtered particles, not the raw particles, during selection in WEB.

• Verify the windowed particles

  This step is optional if using classification-based verification later on.
  Sort through the output files to eliminate any non-particles. This step is optional if using classification-based verification later on.
  Sorting can be done in WEB using the Categorize from sequential montage operation. When the particle image files are displayed on the screen, use the mouse to click on each good particle. For each micrograph, save your selections to a file called good/good***. (More details in the SPIDER FAQ and partpick.html). This creates:
  

  ¤

  good/good***:

  A set of particle selection doc files, one for each micrograph.

• Remove any duplicated particles and list number of picked and selected particles by micrograph

  The procedure: renumber.spi creates:
  

  ¤	good/ngood***:	A set of particle selection doc files, one for each micrograph, listing unduplicated selected particle numbers.
  ¤	percent_selected:	A doc file listing percentages of automatically picked vs manually selected particles for each micrograph and overall.

  When manually selecting particles in WEB, sometimes a particle is accidentely double clicked. This procedure will delete the appropriate lines from the file. Run this procedure, even you you don't think you made any mistakes - subsequent procedure files will require the ngood*** files as input.

• Create lists of images in each defocus group

  sel_by_group.spi reads ../Power_Spectra/def_avg, ../Power_Spectra/sel_group, and ../Particles/good/ngood**** to create selection document files and a group summary file:
  

  ¤	sel_particles_***:	A doc file for each defocus group, listing all the particles in that group.
  ¤	sel_group:	A summary doc file with 4 columns listing: Defocus group, Number of particles, Cumulative number of particles, and Average defocus value for each group.

  Note: this step is not necessary when using Nature Protocols data set.

• Create stacks of images for each defocus group

  win2stk.spi reads sel_group, creates stacks:
  

  ¤

  data***:

  A stack file for each defocus group, containing all particle images from that group.

  Note: this step is not necessary when using the Nature Protocols data set.

• Create reference projections for alignment from a reference volume

  Reference projections (images) are views of the object at known angles. In what follows, angular accuracy is set to 15 degrees, which results in 83 reference projections. A reference volume is needed in the top level project directory to create the projections. The row and column dimensions of the reference volume must match the dimensions of the particle windows. (see resizevol.spi)

  refproj.spi reads ../reference_volume and sel_group to create two sets of files:
  

  ¤	refangles:	A doc. file from VO EA listing the three Euler angles for each of the projections.
  ¤	projs/prj_****:	Stacked reference images from PJ 3Q corresponding to the angles in the preceeding file.

  There are two types of reference projections:

  1. You can use the same stack of references for all defocus groups, by setting register [usectf] = 0. They will be in the file: projs/prj_001
  2. Generate more realistic references by multiplying them by the CTF There will be a stack of reference projections for each defocus group.

• Align particles to the reference projections

  1. If you are using the same reference stack for each defocus group:
     apsh.spi reads projs/prj_***, refangles, and data*** to create an alignment parameters document file and a stack of the aligned images.
     

     ¤	align_01_001:	Shift and rotation parameters for the best matched projection.
     ¤	dala01_001:	Aligned stack of the particle images.

  2. If you have references for each defocus group:
     apshgrp.spi reads sel_group, projs/prj_***, refangles, and data*** to create a set of alignment parameter document files and stacks of aligned images.
     

     ¤	align_01_***:	Shift and rotation parameters for the best matched projections.
     ¤	dala01_***:	Aligned stacks of the particle images.

     If you are using Publish and Subscribe on a Cluster, activate PubSub within apshgrp.spi procedure heading.

• Make particle selection document files listing particles cooresponding to each reference projection

  select.spi reads ../Alignment/sel_group, and ../Alignment/align_01_*** to create the following doc files:
  

  ¤	df***/how_many:	Lists number of particles associated with each reference view for each group.
  ¤	df***/ref_sel???:	Lists the particles numbers associated with each reference view for each group (from 'VO MQ').
  ¤	how_many:	Summary file listing total number of particles associated with each reference view.

• Create average images for all projection groups and combine all alignment parameter files into a single summary file (if necessary).

  average.spi reads df***/how_many, select/sel***, ../Alignment/sel_group, and ../Alignment/dala01_*** to create the following files for all defocus groups.

  ¤	avg***:	Stacks of average images for each set of projections.
  ¤	var***:	Stacks of variance images for each set of projections

The next three operations help to identify a threshold correlation coefficient that describes true particles as opposed to any erroneously selected particles. Create histograms of each defocus group that plots the number of particles vs. the cross correlation value. For example, if the histograms display a bimodal distribution, the higher peak may correspond to actual particles, while the lower peak may show noise. The threshold should be chosen between two such peaks.

• Identify a threshold correlation coefficient for selecting true particles.

  Create histograms for each defocus group that plots the number of particles vs. the cross correlation value. If the histograms display a bimodal distribution, the higher peak may correspond to actual particles, while the lower peak may show noise. The threshold should be chosen between two such peaks.

  cchistogram.spi reads ../Alignment/sel_group, ../Alignment/align_01_***, and align_01_all to to create histograms and a gnuplot control script:

  ¤	hist/cchist_***:	Set of histogram doc files of cross correlation values by defocus group.
  ¤	hist/cchist_all:	Overall histogram doc file of cross correlation values.
  ¤	gnuplot_cchist_***:	Gnuplot scripts to display group cross correlation values histograms.
  ¤	gnuplot_cchist_all:	Gnuplot script to display overall cross correlation values histogram.

  To view the overall histogram plot issue following system command: gnuplot -persist gnuplot_hist_all.ext
  

• Compute correlation thresholds for discarding particles

  ccthresh.spi reads ../Alignment/sel_group, ../Alignment/align_01_***, and hist/cchist***. Using the given percent cutoff level, it creates:
  

  ¤	thresh	Doc file listing CC Value thresholds for each defocus group.
  ¤	sav_particles_***	Particle selection files listing particle numbers of images with correlation values above the threshold,sorted by correlation value..
  ¤	rej_particles_***	Particle selection files listing particle numbers of images with correlation values below the threshold, sorted by correlation value.

  E.g., if a threshold of 0.20 is selected, then 20% of all particles in each defocus group having the lowest correlation values will be rejected. The output doc file thresh lists, for each defocus group, the cutoff correlation threshold, and the numbers of particles above and below the threshold. NB, BECAUSE THE HISTOGRAM 'BINS' THE VALUES, THE NUMBERS OF PARTICLES ARE NOT EXACT, AND THEREFORE WILL NOT EXACTLY MATCH THE OUTPUT OF dftotals.spi.

  Use the Montage from doc file operation in WEB to display the rejected particles identified in the selection doc file (e.g. rej_particles_001). The greater the correlation coefficient value, the more similiar the particle should look to the reference projection. This will help you determine whether the correlation coefficient threshold that you chose is appropriate for selecting true particles. Select Compute Average to view an average of the particles displayed.

  The correlation thresholds in thresh may be edited at this time, if you wish to apply different cutoff thresholds.

• Make final selection of particles above the correlation thresholds

  dftotals.spi reads ../Alignment/sel_group, ../Alignment/align_01_***, and thresh. It applies the updated cutoff levels from thresh to create:
  

  ¤	sel_particles_***	Particle selection files.
  ¤	sel_group_cclim	Group selection file with updated numbers of particles in each group.

  The particle selection files list particles with correlation values greater than the threshold and are used to select particles for inclusion in the reconstruction.

• Check the spatial distribution of views (angular directions) for the reconstruction.

  plotrefviews.spi reads ../Alignment/sel_group, how_many, and df{***[grp]}/how_many. It creates script files of gnuplot commands:
  

  ¤	gnuplot_view	Gnuplot script to plot overall histogram of number of particles vs projection view.
  ¤	display/gnuplot_views_***	Gnuplot scripts to plot histogram of number of particles vs projection view for each defocus group.

  plotangs.spi and its procedure plotangs_p.spi read ../Alignment/sel_group, ../Alignment/refangles, and df***/how_many to create:
  

  ¤	display/angdis_by_group:	A stack of SPIDER images showing distribution of sample images among various reference projections for each group.
  ¤	display/angdis_all_groups:	Combined SPIDER image file showing distribution of sample images among various reference projections for all groups.

  These images show the various angular reference groups represented by small circles. The radii of these circles are proportional to the number of particles in each group.

  Use the Montage operation in WEB to display the images.

• [Optional] Limit strongly over-represented angular projections

  bestim.spi reads ../Alignment/sel_group, ../Alignment/refangles, df***/how_many, and df***/ref_sel{***[grp]} to create:
  

  ¤	sel_group_cclim	Group selection file for specified defocus group
  ¤	sel_particles_***	Particle selection file for specified defocus group

  The selection limits the numbers of images per reference view direction to a certain specified count. This will limit strongly over-represented angular projections which may cause a shape error in the reconstruction. Continue reconstruction with the limited set of particles.

• Split the particle images, create reconstructions, merge reconstructions into final reconstruction.

  Split the particle images from each defocus group into two sets, create paired reconstructions, determine resolution for each reconstruction, merge group reconstuctions into final combined reconstruction and compute reconstruction resolution. Choose one of the following two procedures

  1. For normal serial reconstruction use recon.spi.
  2. For parallel reconstruction under control of PubSub use pub_recon.spi and recon.spi.

  These procedures read sel_group_cclim, sel_particles_***, ../Alignment/dala01_***, and ../Alignment/align_01_***. They create two subset reconstructions for each defocus group using 'BP RP 3' for back projection, compute the resolution of each defocus group reconstruction by comparing the two volumes, and create a CTF correction file for each group:

  ¤	df***/vol01_sub1:	Reconstructed volume from first subset of images for each defocus group.
  ¤	df***/vol01_sub2:	Reconstructed volume from second subset of images for each defocus group.
  ¤	df***/fscdoc:	Doc files containing FSC curve for each defocus group.
  ¤	df***/ctffile:	CTF correction file for each defocus group:

  Finally, they apply CTF correction to all the defocus group subset volumes (see the TF CTS operation), add together the two subset volumes to make a volume for the entire data set, . and calculate the FSC resolution curve for the combined reconstruction (see FSC operation) creating the following files:

  ¤	vol01:	3D reconstruction from entire set of particles.
  ¤	combires:	Doc file containing FSC curve for final reconstruction.
  ¤	resolution:	Doc file containing group number, normalized frequency, and resolution (in Angstroms).

  This reconstruction is known as the initial volume and will be used as the input for the Refinement step.

• Check the defocus group volumes by viewing a central slice from each volume

  slices.spi reads sel_group_cclim, and df***/vol01_sub1 to create:
  

  ¤

  central_slices:

  Stack of central slices from each first subset defocus group volume.

  Use the Montage operation in WEB to view the stacked slices. Ideally, each image should show a particle isolated from its surroundings, exhibiting some internal structure - somewhat like the reference images. If any images are blank or otherwise unusual, go back and check the processing of that defocus group.

• Check the FSC curves from the group volumes by viewing a plot of the curves

  plotres.spi reads sel_group_cclim, combires, and df***/fscdoc. It creates a text file of gnuplot commands:

  ¤

  gnuplot_res:

  Gnuplot script for plotting the resolution curve of each defocus group, along with the combined resolution curve.

  

  Examine this plot and see if there are any unusual curves, if so, go back and check the processing of that defocus group.

• Filter the initial volume

  Information at spatial frequencies above the cutoff value is considered to be uncorrelated noise, and should be filtered out.
  filt.spi reads resolution and vol01. It filters the initial volume at the cutoff frequency to create:
  

  ¤

  volfq01:

  Filtered volume.

  Use the Surface operation in WEB to view the filtered volume.

• The refinement step uses the following input files ('***' denotes group number and '##' denotes iteration number

  ../params:	Parameter definition file.
  ../Reconstruction/sel_group_cclim:	Group selection file.
  ../Reconstruction/vol01:	Initial starting volume.
  ../Reconstruction/sel_particles_***:	Particle selection files.
  ../Alignment/align_01_***:	Initial alignment parameter files.
  ../Alignment/data***:	Unaligned stacked image files.
  input/scattering:	Optional amplitude enhancement scatter file.
  input/mask:	Optional amplitude enhancement mask file.

• Edit the values in refine_settings.pam to set necessary values for refinement parameters and specify any non-standard input file names.

• If you are using PubSub, copy latest SPIDER executable into the current directory and rename it: spider. e.g.
  cp /spider/bin/spider_linux_mp_opt64.18.16    spider

  This ensures that the same SPIDER executable is available throughout whole refinement.

• Run either: refine.pam (If using normal serial operation) or pub_refine.pam (If using parallel refinement on a cluster with PubSub)

  There are a number of sub-procedures used in refinement: refine_settings.pam,   prepare.pam,   enhance.pam,   grploop.pam,   smangloop.pam,   mergegroups.pam,   endmerge.pam,   endrefine.pam, and  

  If you are running on a cluster using PubSub you will also need: publish.perl,   pub_refine_start.pam,   pub_refine_doc_sync.pam,   pub_refine_wait.pam,   pub_ref_loop_clone.pam,   pub_ref_loop_declone.pam,   pub_ref_merge_clone.pam,   pub_ref_merge_declone.pam, and pub_fixrefine.pam.

  Further details are given in the Refinement page.

  Among the outputs of refinement are volumes, images, and document files. Some of the important outputs are listed here ('***' denotes group number and '##' denotes iteration number):

  ¤	final/align_##_***:	Alignment parameter doc files
  ¤	final/fscdoc##_***:	FSC curve doc. file
  ¤	final/val##:	Unfiltered volume
  ¤	final/vol##:	Filtered volume from all images
  ¤	final/vol##_sub1:	Filtered volume from first subset of images
  ¤	final/vol##_sub:	Filtered volume from second subset of images
  ¤	final/dbpr##_***:	Resolution curve doc file
  ¤	final/bpr##_***:	Group volume
  ¤	final/bpr##_***_sub1:	Group volume from first subset of images
  ¤	final/bpr##_***_sub2:	Group volume from second subset of images
  ¤	final/bpr##:	Final volume from all images
  ¤	final/bpr##_sub1:	Final volume from first subset of images
  ¤	final/bpr##_sub2:	Final volume from second subset of images
  ¤	final/ofscdoc##:	Overall FSC curve doc file
  ¤	resolutions:	Doc file listing resolution at each iteration.

• Plot the refinement FSC curves

  The procedure plotrefres.spi reads input/sel_group, final/resolutions, dres##, dbpr##_***, and dbpr##. It creates two text files of gnuplot commands:
  

  ¤	gnuplot_refi:	Plots combined resolution curve for each iteration of refinement.
  ¤	gnuplot_refd:	Plots resolution curve for each defocus group for the last iteration.

• [Optional] View angular data output from Refinement procedure files

  The procedure angdisp.spi reads final/align_##_*** and creates angular distribution images:
  

  ¤

  disp_ii_***:

  Set of images showing angular positions from the align_ii_*** refined alignment parameter output files.

  The procedure angdiff.spi reads final/align_##_*** and creates:

  ¤

  diff_ii_***:

  Image showing changes in angular positions from iteration n-1 to iteration n.

---

Other Useful Methods

Create tar files for backup

backup a shell script, puts a number of vital files into tar files, which can be backed up on tape. Edit the shell script to correspond to your data extension, make sure it is executable and run it from your project directory. It creates 2 tar files order.tar, and project.tar

Difference Maps

Differences between volumes can be assessed by subtracting one volume from the other.

Frequency amplitude correction with X-ray scattering data

Enhance the Fourier amplitudes of a reconstructed cryo-EM volume so they more closely resemble those of experimental low-angle X-ray scattering data.

Classification of Particles

This method can be used to investigate differences between particles.

See more methods on the techniques page

---

References

1. Spherical Deconvolution Improves Quality of Single Particle Reconstruction.     Kishchenko GP and Leith A.    J. Structural Biol. 187: 84-92 (2014).   
2. Use of SPIDER and SPIRE in Image Reconstruction .    Leith A, Baxter W and Frank J.    International Tables for Crystallography.    Vol. F, ch. 19.8, 620-623 (2012)   
3. Fundamentals of three-dimensional reconstruction from projections.     Penczek PA.    Methods in Enzymology 482: pages 1-33 (2012) .   
4. Prevention of overfitting in cryo-EM structure determination.    Scheres SHW and Chen S.    Nature Methods. 9: 853-854 (2012)   
5. SPIDER image processing for single-particle reconstruction of biological macromolecules from electron micrographs.     Shaikh TR, Gao H, Baxter WT, Asturias FJ, Boisset N, Leith A, and Frank J    Nature Protocols. (online) 3: 1941-1974 (2008)   
6. Three-Dimensional Electron Microscopy of Macromolecular Assemblies     Oxford University Press, (2006)   
7. A cryo-electron microscopic study of ribosome-bound termination factor RF2.    Rawat UBS., Zavialov A, Sengupta J, Valle JM, Grassucci R, Linde J, Vestergaard B, Ehrenberg M, and Frank J.    Nature 421: 87-90 (2003)   
8. Ribosomal dynamics explored by cryo-electron microscopy.    J Frank    Methods 25: 309-315 (2001)   
9. Three Dimensional Reconstruction with Contrast Transfer Compensation from Defocus Series.     Penczek PA, Zhu J, Schröder R, and Frank J.    Scanning Microscopy 11: 147- (1997)   
10. SPIDER and WEB: Processing and Visualization of Images in 3D Electron Microscopy and Related Fields.     Frank J, Radermacher M, Penczek P, Zhu J, Li Y, Ladjadj M, and Leith A.    J. Structural Biol. 116: 190-199 (1996)   
11. The ribosome at improved resolution: New techniques for merging and orientation refinement in 3D cryo-electron microscopy of biological particles.     Penczek P, Grassucci RA and Frank J.    Ultramicroscopy 53: 251-270 (1994)   
12. Three-dimensional reconstruction of single particles embedded in ice.    Penczek P, Rademacher M, and Frank J.    Ultramicroscopy 40: 33-53 (1992)   
13. SPIDER -- a modular software system for electron image processing.    Frank J, Shimkin B, Dowse H.    Ultramicroscopy 6: 343-358 (1981)   

---

Source: mr.html     Page updated: 8/5/14     ArDean Leith