1. Trouble Shooting | Home

Some of the more common problems are covered here. If you cant solve a problem in working hours contact your local contact, outside working hours try the EHCs on 8787

Contents

1. Restarting the GDA

To start GDA: From Activities choose a terminal from the proposed icons and type gdalog, then open another terminal where you type gdaservers. Once you see the message that servers have started type on the same terminal gdaclient.

In recent GDA versions there are two Clients, one is for running measurements and the other for microscope and stage control and for setting the energy. When exiting GDA you need to exit both clients, and even then, the command window where you would re-type gdaclient will be very slow in returning the command prompt. To force GDA exit, press Ctrl+C on the command window and the prompt will become available. Alternatively you can start another session of gdaclient on a fresh terminal.

NB: If the command 'gdaclient' in I18 doesn’t open both clients then you manually start the missing one by opening a new terminal window and typing 'mainclient' to get the GUI with Experiment, Plot and Mapping perspectives or 'synopticclient' to get the VMA stream and live controls.

If the Client does not start at all (usually you get an error dialogue box for this) or if you are not getting all the perspectives looking as they should then reset it in the command window by typing gdaclient –reset. This will force some stuff to go back to default so only use it if you need to (so perhaps trying to start the client for a second time before you resort to resting is a good idea).

2. Mapping won't start /stop

• Map not starting (try a couple of times and then):

If the stage seems ok (green in Epics screens) but you still have issues mapping then there are 3 IOCs that you can restart. Click on Hardware Status on the bottom left of the Synoptic screen. At the bottom right (see image below, red circle) click on Restart IOCs. This will restart 3 IOCs and will let you know when it is done If the any of the Stage motors is Red then go to section 3.

Open

Fig 1: Hardware Status / IOCs

• If a map will not Stop :

On Synoptic click on Trajectory on the screen that comes up click Abort. After this restart IOCs as described above.

Open

Sometimes a map that has failed is still semi-active in the queue (so stuck in Resumed mode and the % is not 100) so it prevents you from running another one. Try stopping it and if this fails then : In GDA’s search bar (Quick Access on the top right) type and select Processing Queue. This produces another queue which you can clear by right click on any scan and selecting Clear Queue. NB: this will clear everything from the queue so if there is any information from previously ran scans that you want to retain, do this first and then Clear.

3. Stage Issues

To diagnose it helps if you have the EPICS (Synoptic) screen of the stage you are using open (T1 or T3).

If a motor goes into Encoder error the black square below motor position will go into red Eloss. Click on it to clear. If the motor is red but no Eloss then move it with + or - (note and edit if required the step size in black next to STOP) .It should now move and get into green state.

Homing Stage Motors

NB: Homing may involve a motor moving to an extreme end of its range so consider possible collisions with detectors etc. Also before homing note down all axes positions of T1 so you can return to your sample.

Open the screens for T1 and under Homing click on STEP-08 axes. From the homing screen and Group axes to home select All and then press Home. There is an Abort button if you need it (ie about to have a collision).

Power-cycling the Geobrick (only after advice from local contact)

First stop the IOC : On Synoptic click on Hardware Status and on the screen that opens find Aerotech sample stages.

Click on proservcontrol and Stop. After you power cycle press Start.

Turn the motor power switch (the black one) to the off position.

Turn both the logic and IO switches (the two white switches) on the front of the blade power supply to the off position. Make sure you get the right ones! The above diagram shows how these switches are usually connected but make sure this is the case.

To apply power repeat the above process in the reverse order.

4. If XANES/XAS scans don't run (or are flat) or acquiring an mca gives you nothing

If acquiring gives you zeros (or XAS scans are suspiciously flat) but you are sure you have beam etc there is an obscure bug that could be responsible. To get past it do a short map as this will kick the detector into action.

If the problem persists (and you are getting error messages in GDA log about tfg or daserver then you need to restart the daserver. On the Synoptic click on Hardware Status (bottom left) and then click on proServControl next to TFG2 da.server (orange circle in Fig 1). On the screen that opens press Restart.

Then restart the GDA - see section 1

5. Maps don't display

if the map seems to be running but you are not getting any live display (also you only have one file instead of two per map in the Mapped data tree) then the Processing server might need restating.

In the Synoptic click on Hardware Status on the bottom left of the Synoptic screen. Click on procServControl next to Processing Consumer (black circle in Fig 1) and on the screen that opens click Restart.

What can also happen is that the elemental maps stop being updated, for example after a beam loss when the map restarts. This could be because the Processing that creates them has timed out. You can confirm the raw data are still been collected by inspecting the raw map (sum entry). The elemental maps can then be recreated offline.

6. Excalibur (Diffraction) Detector

Restarting and House keeping:

To restart the IOCs go from Hardware status to Excalibur ODIN processes and click Restart All. Doing this will turn the Low Voltage off so you need to turn it back on. To get Housekeeping where the voltage setting is go to Synoptic: Excalibur : Status

The bullet proof way of restarting (if you are still having issues after the IOC restart):

Before powering off, stop all IOCs ( Under Hardware status → Excalibur ODIN processes → Stop under ODIN Status)

-Power cycle the detector and wait a minute (Also in Excalibur ODIN processes →  PDU status → Off and then On)

-Check that all lights (but one) are green at the back of the detector. if not repeat power cycling. 

-Start IOCs ( Excalibur ODIN processes → Start under ODIN Status). If Start-Stop-Restart are not available then you need to find on Hardware status the Excalibur IOC restarter and Start it.

-From Synoptic : Excalibur and then Status:  In Housekeeping turn Low Voltage back on.

7. Spikes in FFI0

Assuming this is not due to mono glitches (ie spikes were not there from the begining) first check if FF and I0 separately have the same features. If not then the bug can be rid of by restarting the daserver (Synoptic: Hardware Status: click on proservcontr next to daserver...on the dialogue that opens click restart). Then restart GDA servers and client.

If they persist then they are sample related so try on another one.

8. Microscope Camera Problems

The online camera is controlled by EPICS so if you’ve got problems ie it doesn’t seem to be running or the image is too dark / light then find the VMA screen in the Synoptic :

-If Camera 1 is not live then its State would be “Idle”….Press “Start”…it should now be in “Acquire” state and you should have an image. If it still doesn’t go into “Acquire” then go to Setup menu and try “Reconnect” in Reset Cam field.

-If the image is too dark/light try turning Exposure Auto Mode from “Continuous” to “Off” and then in Acquisition next to Exposure (s) try entering a lower or higher number of seconds.

But keep in mind that if you don’t turn the Auto mode back to Continuous you may have difficulties with subsequent samples and generally it is better to be in Continuous mode.

Also keep in mind that the camera takes a couple of seconds to adjust to large changes of illumination so give it some time to adjust before you decide that Auto Exposure doesn’t work for you.

9. No beam on the sample but storage ring is ok: The feedback is lost

Symptoms

• If the values in the S2 drain current window are all about zero and there is beam in the beamline (shutters are open, current in the ring, etc) then the feedback may have crashed.

Solution

• Open the 'roll fast feedback' and 'pitch fast feedback' windows using the buttons in the S2 drain current panel ('S2' then 'BPM' then 'Current Amplifier' then 'roll' or 'pitch' fast feedback).

• In both these windows : Where it has Feedback "Auto" put it to "Manual". Then turn the feedback "OFF" using the on/off listbox.

• In the DCM screen set the second crystal fine roll and fine pitch to zero by typing 0 in the blue text box in each sub panel, hitting return. Or hitting the fine pitch and fine rool 'to zero' buttons.

• Go back to the 'roll fast feedback' and 'pitch fast feedback' windows. Turn the feedback "ON" and then back to "Auto". This should give you some signal in the feedback S2 drain current window.

The feedback can also be lost when moving filter D6A, if this occurs it may be necessary to move D6 to the 15 m Al value before trying again. Keep that filter in.

10. There is beam on the feedback but not in the ion chambers

As there is a pinhole of 200-400 microns diameter from which the beam goes through before reaching the sample if the ion chamber gets knocked for example when changing samples then you will not have any beam exiting.
The general idea is you have to roughly centralize the ion chamber without the pinhole and then finely centralize with the pinhole back in.
-First have a look of whether the ion chamber seems to be generally aligned with the beam exit which is a round window of kapton on the KB vessel behind the chamber.
Take out the pinhole (pull up the black metal frame at the back of the chamber).
Remove your sample as inspecting readings on the second ion chamber (Ion 2 on the Scaler screen in EPICS ) is needed for the alignment.
Do you now have a reading on Ion1 and Ion2 ?
If not it means the chamber has been moved quite a bit (visual inspection might suggest which direction and by how much) so first you need to manually move it in Epics to get some beam out:

You can move the ion chamber by its EPICS controller, you find it on the Synoptics by clicking the PIN box (between KBM and T1). The current position is the Green value, while the Black value is the step you move by with the + and – relative move buttons. The step value –which is in mm- can be changed by 2ble clicking on black value and entering a new number.

You can start by setting a step of 0.5 and moving about in X and Y until you get beam on both Ion1 and Ion2. Once you get some beam on both chambers you can roughly centralize by finding the ranges that allow beam on Ion2 (as exit hole of the chamber is much smaller than entrance hole) and going in the middle. You can enter the position you want to go to by entering in the box with the blue numbers and hitting Enter.Then you need to do some coarse and fine alignment in GDA after putting the pinhole back in the ion chamber (there is only one way that it can freely slide in the chamber, don't push cause you will move the whole chamber again). Coarse Alignment :You scan the chamber by using the command line in the Jython Console of GDA . The command will look like :scan piny 8 11 0.1 pinx 8 11 0.1 counterTimer01 500 (8 start, 11 end , 0.1 step : for start and end choose +/- 1.5 mm from current positions you get from the EPICS screen). From the produced graphs in Plot choose I0 and It where the centre should be for X and Y. It is also useful to look at the chamber readings outputted in the Console as you will spot numbers going up in chamber 2 (or you can be looking at the Scalers Epics screen).Once you have beam in Ion Chamber 2 you can Interrupt the scan and move to the correct pinx and piny positions with the commands: pos pinx 8.5 andpos piny 8.5or by putting the right numbers in the Epics PIN screen.Fine Alignment: Finally two separate fine scans to centralize each motor :scan pinx 8 8.8 0.02 counterTimer01 500 (scan +/- 0.4 mm for current positions)scan piny 8 8.8 0.02 counterTimer01 500Again take pinx and piny to the correct numbers.

11. OLD: Restarting the CMOS diffraction camera

If the camera doesn't seem to respond try the following :

-On the RHS PC where the camera software is running exit the PSL viewer software

-To restart it double click on either PSL Viewer Time out or No time out (as an example, acquisitions of <10 sec per point would be Timeout but longer would be No timeout)

-From Camera menu load Multiconfig file

-On Synoptics workstation click on Hardware Status

-Find the camera server called PSL IOC and click on the Control button next to it

-Click Restart

-Try an acquisition from GDa's command line to confirm camera is now responding

-If you give up on software restarts you can try a hardware reset: there are two whitish boxes at the end of the experimental table, one with a green and one with a blue LED. Power cycle them and then try again the software restarts

12. If you press the table emergency OFF

if the table emergency stop is pressed this will de-energise a number of motors around the experimental table. These include the sample stage and the motors that move the SiD detector.

So you need to:

1. pull back pressed button

2. press lit blue reset button, light should go out (if it doesn't then the button that was pressed is still not released)

If the detector is close to the sample move it away (+ve SiD X) before continuing to avoid a collision. To do this you need to energise the motors :

3. In the synoptic select hardware status select Hardware Status and click on Pmac next to PMAC-02.

4. select connect…..select energise…find motor 9 and tick ….select send

This re-energises the motors. Now you can move to a more positive SiD X (eg 60) making a note first of where you were for the measurements.

6. Now rehome sample stage motors (see item 4). If you have a bulky cell you may need to consider removing it to avoid collisions.