FAQ : SPIDER & WEB

Contents

QUESTION
Why is SPIDER slow with lots of windowed images?
ANSWER
If you have over a few thousand windowed images then you SHOULD always place the images in a SPIDER stack file.
otherwise there will be severe thrashing of the file system leading to very poor performance of any system operation, e.g. ls, and poor performance of SPIDER. This is not a SPIDER issue, it it a filesystem isssue.

QUESTION
Howto generate a contour plot?
ANSWER
Using the operation CO you choose levels and increments according to the information you are trying to get. The smaller the increments, the more finely you get a sense of the density distribution in your image. Contouring raw, noisy images makes no sense -- you should only apply contouring to average images or highly low-pass-filtered images. You want to pick start and end level normally as the lowest and highest value occurring in the image, so the contour diagram will give you information about the entire value range, unless you want to suppress some of that range. Note also that there is an option to create contouring directly overlayed on the image -- the FC operation. This produces more informative and esthetically pleasing contour maps. To use this option, you must first create a large version of the image using the IPIP operation, otherwise the contours will look very pixelated.

QUESTION
What about large (>2GB) stacks or voumes in SPIDER
ANSWER
With most compilations, SPIDER can create and access disk based files > 2GB. Some compilations on a 32 bit OS may be limited to these small volumes. I suggest you switch to a 64 bit OS if you are having troubles.

QUESTION
Howto obtain the volume of a density map displayed with a given threshold?
ANSWER
Use the HI R operation and set lower range desired threshold THRESH and upper range byond the file maximum. The number of PIXELS IN HISTOGRAM is the volume VOLP in units of pixel**3. To obtain the actual volume, multiply VOLP with the volume of a voxel, which is the (Angstroms per pixel)**3. Note that everytime you change the theshold THRESH, you have to repeat this analysis.

QUESTION
How do I set the number of processors SPIDER uses on a SMP multiprocessor machine using OMP?
ANSWER
Number of processors is set before (or during) a run. There are two methods you can use to control this.

QUESTION
How to import images from GATAN CCD camera into SPIDER?
ANSWER
; -- Procedure to import images from GATAN to SPIDER
DO [num]=1,7 ; LOOPs over the micrograph number
   CP FROM RAW ; Copy operation
   (16) ; How many bits
   mic/mic{***[num]} ; Input file name, indexed by counter
   (1024 1024) ; Input file dimensions
   (8) ; Skip header bytes
   (1) ; Most significant byte
   N ; Do not fold negatives
   mic/spi{***[num]} ; Output file name, indexed by counter
ENDDO ; End of LOOP
EN ; End of procedure

/p>

QUESTION
How do I use a single reference to align my 2-D images?
ANSWER
There are two operations in SPIDER that can be used for a reference-based alignment. Their original purpose was multi-reference alignment, but by limiting the number of reference images to one one can perform a reference-based alignment of 2D data. The operations are called AP SH and OR SH. Both operations are very simple and they do not require any procedure programs. The alignment parameters found can be applied to the data using procedure programs included in the respective manual chapters.

QUESTION
I have a simple 3-D model for my molecule. How can I compare this model with my SPIDER model?
ANSWER
This is done using a 3D projection alignment procedure. It is based on operation AP MD. The corresponding manual chapter contains an example of the whole procedure. It assumes that a SPIDER volume containing a reference structure is available.

QUESTION
How to Determine the resolution of a reconstruction.
ANSWER
Using operation AS R one can obtain averages corresponding to odd- and even-numbered images. (sample procedure file:)
AS R
data/my_images****
1-2200
O
aveodd001
varodd001
aveven001
vaeven001
EN
The averages can be compared using operation "RF". This operation will produce a document file with R-factor, phase residual, and Fourier ring correlation measures.

Alternatively, operation "RF SN" can be used on the image series to obtain spectral-signal-to-noise-ratio measure of the resolution (sample procedure file:)

RF SN
ddata/my_images****
select    ; Selection document file, see the manual file
mask    ; Mask image fine defining region of interest
0.5
ssnr_doc

QUESTION
I have two averaged images that are at two different defoci. How do I correct one to match the other?
ANSWER
Due to a low SNR in 2D image data we do not apply CTF correction on these images. Instead, we correct for the CTF final 3D reconstructions. This is done (for any number of volumes) using TF CTF operation. Fourier volumes containing CTFs for all the defocus sets are needed and they can be created using operation: TF C3. Other CTF operations are grouped under operation "TF".

QUESTION
When the RT 3D operation rotates a volume by 45,0,0 degrees (phi, theta, psi), why isn't the last Z-slice rotated?
ANSWER
This is not an error, it is supposed to work this way. To rotate the volume one has to interpolate it. The 'tri-linear' interpolation requires that for a voxel (i,j,k) the neighbouring voxels are available as well, i.e., voxels (i+1,j,k), (i,j+1,k),... ,(i+1,j+1,k+1). If the neighbouring voxels are unavailable (they are outside of the field or in the corners) the current voxel is left unchanged. Thus, to rotate last slice, k=NSLICE, one would need values at NSLICE+1, which are unavailable. Thus, the last slice is left without change.
If you want to rotate volume around Z axis (angles=(psi,0,0)) you can use the operation RT 3Q, which will rotate all the slices by the same angle around Z axis.

QUESTION
Why does RT SQ produce an "artifact" in slices for some angles.
ANSWER
The 'RT SQ' operation assumes that the 2D image (in your case it is a volume, but 'RT SQ' operates on Z-slices independently, so we can discuss 2D case only) is "circularly closed", or "wrapped around". Assume that pixels are numbered from -N to N (this you can always do by subtracting image center). Thus, according to RT SQ pixels number -N-1 is equivalent to pixel +N. In case of corner pixels and rotation angles close to 45 degrees the corners upon rotation will end up close to the center (if you rotate point (N,N) by 45 degrees it will have coordinates (1.41*N,0), and according to the rule described (-0.41*N,0) - close to the center).

Command "RT SQ" was designed to work for objects that are surrounded by noise. In this case it makes sense to put noise in the corners. For model objects, such as yours, it will create "artifacts". Since upon rotation the corners always stick out beyond the image frame one is free to decide what should be put there.

In the manual, each rotation operation has a remark stating which convention was used. RT 3D, for example, leaves corners intact and will work better for your object. RT B leaves you a possibility to supply your own value that will fill the corners. Please note that only "RT SQ" uses quadratic interpolation.

QUESTION
I have a collection of high-resolution rotary shadowed images of various proteins. Can I use SPIDER image processing and analysis to improve the data gained from these images?.
ANSWER
On the basis of the fact that all images represent equally and isotropically treated particles, it should be possible to use alignment algorithms provided in SPIDER, and obtain a 2D average. However, 3D information cannot be retrieved -- the retrieval of the 3D profile would require unidirectional shadowing and the use of a filter that Smith and Kistler developed several years ago. This filter is not implemented in SPIDER, but it could be added.

Other than that, SPIDER could be used for all kinds of more or less cosmetic operations such as contrast enhancement and high- or low-pass filtration.

QUESTION
We have come across a curious phenomenon in imaging nucleosomes using cryo-EM. Images of unstained frozen hydrated nucleosomes are dominated by contrast from the peripheral (~100KDa) DNA component, while the central protein assembly (also ~100KDa, but occupying a smaller volume) has a density very close to the background ice. CTF correction has little effect on the contrast.

I had assumed that the differential contrast was related to the high P content of the DNA, but have now used the nucleosome X-ray coordinates and SPIDER's 'CP from PDB' to simulate projection images. These show strong, and approximately equal, contrast from both the DNA and protein components, in other words, no differential contrast is observed.

ANSWER
Currently SPIDER uses only the phase contrast component, transferred with sin(gamma), for simulations of EM images from atomic coordinates. The differential contrast between DNA and protein core in nucleosomes that you see with cryo-EM is produced by the increased amplitude contrast ("anomalous scattering") of the P atoms. When we have a chance to incorporate both amplitude and phase contrast in the modeling procedure, I'm sure you will find a close match between theory and experiment.

QUESTION
How to convert CCP4 files to SPIDER format?
ANSWER
Use COPY FROM CCP4.

QUESTION
Is SPIDER useful in image averaging and 3-D reconstruction for AFM? If I imaged molecules in different orientations on a surface and I could deconvolute the effect of my AFM tip from the image, can I use SPIDER in any way to help to 3-D average or reconstruct the imaged molecules?
ANSWER
There are operations in SPIDER that you can use for averaging or deconvolution. What you get from AFM is surface topology; i.e., a segment of the molecular surface. At best, you can put the entire molecular surface back together from such segments, if you 1) have copies of the same molecule lying in different orientations and 2) have a way to quantitatively determine these orientations. This would be a process similar to putting the shell of an Easter egg together from its pieces. (I'm saying Easter egg, since the normal egg is quite symmetric, but the Easter egg fits the analogy better since it is painted with a hopefully asymmetric pattern).

But you should note that all operations in SPIDER that are related to the determination of orientations or 3D reconstruction address an entirely different problem, namely to interpret and make use of 2D projections of a 3D density distribution.

Therefore, the answer to your question is: yes, you will find SPIDER quite useful in helping you to analyze and process images, but no, it will not help you with the specific problem of Easter egg-type 3D reconstruction.

QUESTION
I want to make astigmatic CTF functions, and am confused by your definitions of "astigmatism and azimuth". The definition of astigmatism that I know has a long and a short axis which define the ellipse, and an angle that defines the orientation of the long dimension of the ellipse relative to your X and Y axes. I don't understand how this relates to the azimuth in your operations which is defined as "the angle, in degrees, that characterizes the direction of astigmatism. The angle defines the origin direction where the astigmatism has no effect."
What are definitions of astigmatism and azimuth in the TF operation?.
ANSWER
In the SPIDER operation, the zero azimuth points into the direction where the astigmatic focus difference has no effect; i.e., 45 degrees away from your definition. This is practical as a definition , since in this case the astigmatism term simply adds to the mean defocus.

QUESTION
I am aligning particles using either AP MD or AP SH. The protein is "white" and there are nanogold labels on the protein that are "black" (essential negative pixel values). My understanding is that SPIDER alignments only pay attention to the white or positive pixel values and not the black negative pixel values? As long as I normalize the images (avg=0, sd=1), is this truly the case? Will I have alignment difficulties from alignment of the small nanogold dots to each other?
ANSWER
That's incorrect. Alignment algorithms take into account the whole density range, without paying attention to the (arbitrary) polarity. Such dots would tend to align to each other, compromising the overall alignment of the protein part.

QUESTION
Is there any operation in SPIDER which allows images at slightly different magnifications to be normalized with respect to each other before image averaging?
ANSWER
Operation 'IP' will re-interpolate an image onto a new grid, so it will help re-scale to make up for a magnification change. Combine it with either 'PD' (pad) or 'WI' (window) to get back to the same dimensions. However, we don't have a single operation that will let you DETERMINE the magnification change. The corresponding mathematical operation is the Mellon transform, which is not implemented. You can however write a simple procedure with a do-loop trying out different scale changes (using 'IP', etc. as explained above) followed by a 'CC C' operation, taking care that you keep the images centered, and that you only compare non-padded portions of the images.

QUESTION
How to import TIFF images into SPIDER?
ANSWER
Many black and white TIFF files consist of 8 or 16 bit raw data preceeded by a fixed length (often 1024 byte) header. They can sometimes be converted to SPIDER format using the CP FROM RAW operation.
CP FROM RAW ; Copy operation
8 ; How many bits
imgin ; Input file name
512 512 ; Input file dimensions
1024 ; Skip header bytes
imgout ; Output file name

QUESTION
How to rescale an image's range to 0 - 255 using AR?
ANSWER
Use AR SCA instead.

QUESTION
What did 'CA SI' do?
ANSWER
CA SI read in the images and passed them through the mask -- it put the passed pixels into one single array (lexicographically ordered), which becomes one record of a sequential file. This sequential file is subsequently used by 'CA S' to do the analysis. As of SPIDER version 10.0 this is all done within: 'CA S' and 'CA SI' is no longer used.

QUESTION
Howto correct for drift in images by Wiener filtering for example?
ANSWER
It isn't worth the effort. The zeros mean that parts of the transform are completely lost. One could of course think of combining pictures that have undergone different amounts (and directions) of drift, to cover all zeros, but it is much easier to do the experiment right and discard the images affected by drift.
I'm not aware of any efforts in the EM field, for the reasons stated. However, I've seen literature on successful drift correction in general image processing applications, but note that in those applications the SNR is normally much higher than in EM images, meaning that more info can be recovered in the vicinity of the zeros.

QUESTION
When using SPIDER marker MK operation as part of a tomographic reconstruction proceedure, how do I deal with a marker that is not visible in a given projection? Can one just not enter data for that particular marker in the offending projection?
ANSWER
Sometimes it is OK to leave a marker out on one or two images, but in that case you must edit the marker document files so that the key number for the markers correspond to same gold particles on all the images. Either renumber the keys or pick a "dummy" for the missing marker, then delete that line.
It is better not to try and get by with missing markers, however.

QUESTION
When using PubSub I sometimes get following message and it quits running:
poll: protocol failure during circuit creation What can I do to prevent this?
ANSWER
You may see this message if you attempt to run too many rsh processes. Depending on the particular Linux distribution, there may be a limit of as few as 40 processes per minute. It is possible to exceed this limit.
To fix this, you can do one of the following:

1. Wait a few seconds between running parallel jobs. You may need to wait up to a minute.

2. If your system uses:/etc/inetd.conf, modify /etc/inetd.conf to allow more processes per minute for rsh. For example, change:
shell stream tcp nowait root /etc/tcpd2 in.rshd
to:
shell stream tcp nowait.200 root /etc/tcpd2 in.rshd

3. If your system uses xinetd, you probably need to set the "cps" field of:   /etc/xinetd.d/rsh     to a large value such as "200 5" for xinet to handle the expected traffic. We haven't tested this, but here's two fields that:    /etc/xinetd.d/rsh     should contain:

service shell
{
cps     = 1000 10
disable     = no
}

After modifying this file, you'll need to restart xinetd using "/etc/rc.d/init.d/xinetd restart".

QUESTION
I'am trying to fit a high-resolution X-ray model into a low resolution EM map. It appears that SPIDER'S WEB is capable of representing surfaces from maps? If so, what kind of map formats does WEB accept? Do you provide software for map conversion, etc? My maps are in 'crystallographer-friendly' formats such as CCP4 and XPLOR.
ANSWER
Yes, but WEB doesn't allow you to do interactive fits. You'd be better off using the package "O", which we use all the time. What we do is make preliminary displays using WEB, then prepare the volume using low-pass filtration as appropriate, then use an operation in SPIDER that makes a "map" file out of it compatible with "O", then display the map in "O" along with structures directly imported from "pdb". We also routinely import "pdb" structures into SPIDER and convert them to densities, then low-pass filter (and, if applicable, apply "CTF") them, and go the same route as for the cryo-volume into "O", if we want to simulate the appearance of the structure as seen in the microscope, and compare it with what we see as extra density.
Yes, both CCP4 and XPLOR data can be imported.

QUESTION
How can I create a surface model from a SPIDER volume.
ANSWER
One way of visualizing surfaces contained in a SPIDER volume is operation. This allows the user to generate a shaded rendering at a particular threshold level within the volume.

To make a surface model our usual procedure is to use NAG's Iris Explorer to read SPIDER volumes, threshold the volume, and create a surface model in Inventor format. We then use a rendering viewer available in Iris Explorer to render the surface models. This method requires the Explorer module for reading SPIDER images which is free from us. It also requires that you purchase Iris Explorer from NAG.

Another user suggests converting the SPIDER volume to SGI-RGB images with the SPIDER operations PS Z and CP TO SGI and then importing the volumes images into SYNU for surface rendering.

QUESTION
How to pick particles in SPIDER/WEB?
ANSWER

See: particle picking techniques page for recent info. The following older info may also be of some interest:

  1. Low pass filter the particles beforehand, with pfilt.spi

  2. Make sure you have previously run pnums.spi and have the doc file order handy. It lists the number of particles in each micrograph: ;spi/hcc 05-AUG-2004 AT 13:08:03 order.hcc
    ; /   CUM.TOTAL,   # PART,   MICROGRAPH  FIRST   LAST
    1   5   1659.0     1659.0     1.0000   1.0000   1659.0
    2   5   3585.0     1926.0     3.0000   1660.0   3585.0
    3   5   5233.0     1648.0     4.0000   3586.0   5233.0
    4   5   6903.0     1670.0     6.0000   5234.0   6903.0

  3. Create the directory /Particles/good

  4. run WEB

    1. Command/Categorize from sequential montage
      input file: Don't use the filter! It will try to read in tens of thousands of filenames. Instead, go to the bottom line and type in the file name: ....../Particles/win/ser00001.acn
      last file: should be the first file number + about 50 (a screenful of particles).
    2. (then the particles load)
    3. When it asks for a document file, enter "good/good015", where 015 is the micrograph number.
    4. For each new good doc file (i.e. when you start the next micrograph), the particle number starts at 1.
    5. Then pick particles by clicking. If you make a mistake you have to later edit the good*** document file.
    6. When you're done : click the right mouse button.

    some useful options:
    Options/Image/size reduction x2
    Options/Color/Foreground = green
    Options/Contrast - adjust to see particles better

    Then for the next set of 50 particles,

    1. Command/Clear
    2. Command/Categorize from sequential montage
    3. Add 1 to the start number
    4. Add 50 to the last number
    Keep going until you reach the number of particles in that doc file (e.g. 1659 in the first micrograph above). Then start a new doc file with the startkey reset to 1. But the first particle file should be (last file + 1, e.g., 1660 above).

    Check the startkey - reset it to 1 if you're starting a new document file.

Notes:

WEB determines the max & min of the file and writes these values to the header, so you need write permissions on the win/ser* particle files.

The document file produced by WEB has a key column + 2 data columns:
; acn/acn good/good001.acn Fri Sep 29 09:52:15 2000
;key     -    particle#     category# (1 = good particle)
0001     2     1.000000     1.000000
0002     2     3.000000     1.000000
0003     2     5.000000     1.000000
0004     2     7.000000     1.000000
0005     2     9.000000     1.000000
0006     2     11.000000     1.000000
0007     2     13.000000     1.000000
0008     2     15.000000     1.000000
0009     2     17.000000     1.000000
0010     2     19.000000     1.000000
0011     2     21.000000     1.000000
0012     2     22.000000     1.000000
The key is numbered consecutively (= number of lines = N Particles) while the particle numbers skip over the nonselected particles

QUESTION
Howto view picked particles from low defocus micrographs using WEB?
ANSWER
Confirming particles in a gallery of raw windowed files is a pain at low defocus. I suggest using the FQ operation (with filter option: 7) on the raw files first, and play around with various filter parameters. We use the parameters: passband 0.005, stopband 0.30. This results in better visibility, for particle picking. The optimum parameters will probably vary depending on microscope, scanning, and defocus.

QUESTION
We have a problem with the color display of menu text in WEB, because the menu text and background are often almost totally illegible - ie two very similar shades of green, or light yellow on white. Do you have any way of controlling the text and background colour of the menus in WEB on SGI machines (Indy, Indigo2, O2) and on X terminal displays to these machines?
ANSWER
WEB accepts standard Xt Command line Arguments Arguments and uses the Xt resource file (typically the .Xdefaults file in your home directory). Some of these variables can be used to set forground and background for various widgets in WEB. I suggest that you experiment with resource settings in your .Xdefaults file:
e.g.
Web*background : green
Web*foreground : red

or with Web command line options:
web -bg Background-color DAT
web -bd Border-color DAT
web -fg Foreground-color DAT

e.g.
web -bg green DAT
web -bg purple -fg red DAT

QUESTION
Howto use 8 bit WEB on Linux without permanently switching your only display to run in 8 bit mode?
ANSWER

Dr. David Bhella (MRC Virology Unit,Glasgow) suggested a work-around for using Web that allows full operation. Basically it specifies a virtual terminal with an 8-bit display and one with a 24-bit display that can be switched between using ctrl-alt-f8 & ctrl-altf7. Configuration is fairly straight forward and works on some flavors of GNU/Linux. Instructions.

QUESTION
Howto view SPIDER format images without using Web?
ANSWER

You might be able to use the "CP TO RAW" operation in SPIDER to convert a SPIDER format image to a 8-bit "raw" image. Most imaging softwares allow you to input a "raw" image if you specify the image size. If you choose to do this, you can deal with basic display functions but may not easily emulate many interactive functions contained within Web.

Source file: faq.html     Updated: 21 Jan 2010     ArDean Leith