...
- The two FMC cards must be placed in the AMC 525 carrier in the correct slots, with the Digital I/O card in slot 0 and the FMC 500 in slot 1. See the image below to verify the correct configuration:
Note that in order to avoid an I2C collision with an AMC temperature sensor (and thus a MTCA IPMI alert which won't go away) it is necessary to cut a line on the Digital I/O card (is this documented anywhere? - When placing the AMC cards in the crate it seems that it is necessary to be careful about which cards are (logically) adjacent to the processor card. To avoid problems, at DLS we have install the processor card in slot 6 and the carrier cards in slots 2 and 3 (note that these slot numbers don't correspond to the PCIe addresses which we'll encounter later). Specifically, it seems that with the processor card in slot 6 we need to avoid placing a carrier card in slot 5, as otherwise the processor card tries to boot from non-existent mass storage in this slot!
- At this point we recommend its a good idea to ensure that all serial ports are configured with the same data rate. The default line speed is 115200 8N1 for all ports except for the CPU console which defaults to 9600 8N1.
- Now e-keying must be configured.
...
These are all SSMC sockets. The connectors should not be tighter than finger tight, our experience is that over-tightening can use the inner conductor to lose contact. The official documentation (see page 26) recommends 0.2 Nm coupling torque.
| Port | Dir | Signal | Description |
|---|---|---|---|
DAC 1 OUT- DAC 1 OUT+ | Out | ±1V into 50Ω, DC to 500 MHz | Channel 1 differential output. Outputs are driven directly by a LMH6554 amplifier. Unused outputs should probably be terminated into 50Ω. |
DAC 0 OUT- DAC 0 OUT+ | Out | Channel 0 differential output. | |
| CLK IN | In | ±0.3 to 3.3V into 50Ω, 250 to 500 MHz. | Machine RF clock. |
| CLK OUT | Unused (except for development and debugging) | ||
| EXT TRG | In | 0 to 3.3V into 50Ω, threshold at 1.2V. | To ensure synchronisation with the machine revolution a fast rising edge is expected at machine revolution frequency. This signal is synchronised to once during startup, and is sampled continuously thereafter. |
| ADC 0 | In | ±1V into 50Ω, DC to 250 MHz | Channel 0 single ended input. |
| ADC 1 | In | Channel 1 single ended input. | |
System Software Setup
At this point the MTCA crate and all cards should be in a position to power up. Now tangle with the BIOS and install your choice of Linux (we use RHEL 7 for this at DLS). No special drivers (except for the MBF driver, we'll get onto that) need to be installed, but you'll probably want IPMI administration configured.
...
The details of how to manage driver installation are somewhat distribution specific. Once the module has been installed it should be automatically picked up when the FPGA is loaded.
Bringing up the FPGA
- Michael Abbott The automation tools here need to be modified to be generic and the instruction here fixed
First of all the FPGA must be loaded onto the AMC card. The following commands ./load_fpga script does the work: this must be run on the target system will do the necessary workAMC720 processor card:
| Code Block | ||
|---|---|---|
| ||
BIT_FILE=amc525_lmbf.bit ip=192.168.40.200 scp "$BIT_FILE" root@$ip:/tmp./load_fpga -f path/to/amc525_lmbf.bit ssh -x root@$ip amc525_lbtool fpga_load /tmp/amc525_lmbf.bitbit |
(The -f bit-file argument can be omitted if the bit file is in the working directory.)
At this point run lspci -v, and you should see something like the following:
...
- Michael Abbott Need to capture the above on a freshly booted system to correctly show unreachable registers
If we see this then the FPGA has been successfully loaded and the PCIe link is working correctly. Alas, the PCIe IO memory cannot be mapped at this stage because it is too late for the BIOS to identify it. Now reboot the processor card and re-run lspci -v and we should now see:
...
Now the IOC can be started, either by running iocs/$ioc_name or preferably ./start_ioc $ioc_name to run the IOC under procServ.
...
Initial State
When MBF is started for the first time, or when the state file is deleted, the initial state of nearly all PVs is set to zero. Before trying to use the system the input and output compensation filters should be reset to passthrough and the control gains should be set to 1. The epics/initial-state script will perform this operation:
| Code Block | ||
|---|---|---|
| ||
epics/initial-state $ioc_name |
Also note that every time MBF is restarted, the DAC output is disabled. This is deliberate to avoid accidentally driving onto the beam after a restart, but means that after every restart the :DAC:ENABLE_S pvs must be manually set.
Helper Tools
Restarting the IOC
The IOC can be restarted at any time. If however the crate is power-cycled then the procedure is slightly more involved:
- Power on crate, wait for PC to boot
- Reload FPGA with
./load_fpgascript run on PC - Reboot PC (without power cycling) to pick up full PCIe functionality
- Start IOC
- Renable DAC outputs, rearm triggers