Versions Compared

Version Old Version 14 New Version 15
Changes made by

Benoît Roche

Benoît Roche

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: update with new PCIe firmware

...

By default AMC720 and UTC002 are configured for PCIe x4 operation.  The MBF can be operated in this mode, but it will not be optimal in term of data transfer speed.

...

PCIe bus is managed by the PCIe Fabric DA135 installed on the UTC002.  By default the DA135 is configured to handle x4 lanes PCIe link.  You have to upgrade its configuration to be able to use 8 lanes PCIe.

...

Then you need to get the file DA135-PCIE-x8-copper-1.24.0.tgz from Vadatech Products Download Center Portal under Switches / UTC002 / PCI Fabric / DA135 PCIe Gen3 Virtual Switch FRU Information.

Copy the file in /upgrade (from main CPU using scp, its IP address is 192.168.40.1 here) and upgrade the system (you have to use the /upgrade folder as shown below):

Code Block
languagebash
[root@vtipmi ~]# cd /upgrade
[root@vtipmi upgrade]# scp 192.168.40.1:/your_folder/DA135-PCIE-x8-1.2.0.tgz ./
[root@vtipmi upgrade]# tar xvzf DA135-PCIE-x8-copper-1.24.0.tgz
DA135-PCIE-x8-copper-1.24.0
DA135-PCIE-x8-copper-1.24.0/vtpcie_ekeyeeprom.confpex
DA135-PCIE-x8-copper-1.24.0/da.confupgrade
DA135-PCIE-x8-copper-1.24.0/upgradefruDA.xml
DA135-PCIE-x8-copper-1.24.0/systemplx.inieeprom
DA135-PCIE-x8-copper-1.24.0/versionsystem.ini
DA135-PCIE-x8-copper-1.24.0/plx.eepromversion
DA135-PCIE-x8-copper-1.4.0/fruGenMCMC
DA135-PCIE-x8-copper-1.24.0/fruDAredriver.xmlbin
DA135-PCIE-x8-copper-1.24.0/fruGenMCMCvtpcie_ekey.conf
DA135-PCIE-x8-copper-1.24.0/eepromda.pexconf
[root@vtipmi upgrade]# cd DA135-PCIE-x8-copper-1.24.0
[root@vtipmi DA135-PCIE-x8-copper-1.24.0]# vtipmi stop
[root@vtipmi DA135-PCIE-x8-1.2.0]# ./upgrade
Backing up /etc/da.conf to /etc/da.conf.bak
Backing up /etc/vtpcie_ekey.conf to /etc/vtpcie_ekey.conf.bak
Creating Daughter Card FRU inventory
warning, no module power information found.
warning, no zone3 compatibility information found.
Updating EEPROM at address 0x50. Please wait ... (try 1)
Verified data in EEPROM 0x50
Erasing the EEPROM...
Resetting the swtich...Done
Writing to EEPROM: Done

Successfully configured daughter card.  Please power cycle
for the changes to take affect.

[root@vtipmi DA135-PCIE-x8-1.2.0]# vtipmi start

Check Configuration

You have to power cycle the crate for the changes to be taken into account.

...

Stopping MCMC
Stopping Carrier Manager
Stopping Shelf Manager
Stopping failover agent
Stopping file monitor
Stopping SNMP server
Stopping SNMP trap handler
Stopping ethernet monitor
Stopping execHelper
Stopping Memory monitor
Stopping vtipmi custom
[root@vtipmi DA135-PCIE-x8-copper-1.4.0]# ./upgrade
Backing up /etc/da.conf to /etc/da.conf.bak
Backing up /etc/vtpcie_ekey.conf to /etc/vtpcie_ekey.conf.bak
Creating Daughter Card FRU inventory
warning, no module power information found.
warning, no zone3 compatibility information found.
Updating EEPROM at address 0x50. Please wait ... (try 1)
Verified data in EEPROM 0x50
Erasing the EEPROM...
Resetting the swtich...Done
Writing to EEPROM: Done
Programming PCIe Redriver Configuration EEPROM...

Successfully configured daughter card.  Please power cycle
for the changes to take affect.
[root@vtipmi DA135-PCIE-x8-copper-1.4.0]# vtipmi start
Starting execHelper
Starting heartbeat monitor
Starting watchdog timer
Starting memory monitor
Starting MCMC
Starting Managers
Starting trap handler
Starting vtipmi custom

Check Configuration

After DA135 upgrade you can check that pcie command's output changed:

Code Block
languagebash
[root@vtipmi ~]# pcie
                    Virtual           Link     Max    Link       Link
Device   Physical   Switch  Status    Width    Width  Speed      Type
--------------------------------------------------------------------------
1        8          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
2        2          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
3        0          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
4        6          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
5        4          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
6        14         0       Enabled   No Link  x16    2.5 GT/s   Endpoint
7        9          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
8        3          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
9        1          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
10       7          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
11       5          0       Enabled   No Link  x16    2.5 GT/s   Endpoint
12       15         0       Enabled   No Link  x16    2.5 GT/s   Endpoint
13       12         0       Enabled   No Link  x16    2.5 GT/s   Endpoin

However you have to reboot UTC002 (just type 'reboot').  Log back on the UTC002 and pcie command will give the correct output:

Code Block
languagebash
[root@vtipmi ~]# pcie
                    Virtual           Link     Max    Link       Link
Device   Physical   Switch  Status    Width    Width  Speed      Type
--------------------------------------------------------------------------
1        8          0       Disabled  No Link  x8     2.5 GT/s   Downstream
2        1          0       EnabledDisabled  No x8Link       x8     82.05 GT/s   Downstream
3        0          0       Disabled  No Link  x8     2.5 GT/s   DownstreamUpstream
4        5          0       Disabled  No Link  x8     2.5 GT/s   Downstream
5        4          0       Disabled  No Link  x8     2.5 GT/s   Downstream
6        13         0       EnabledDisabled  No x8  Link     x8     2.5.0 GT/s   UpstreamDownstream
13       12         0       Enabled Disabled  No Link  x8     2.5 GT/s   Downstream

...

Downstrea

PCIe link can be x8 at maximum.  It FPGA image is loaded the output of pcie will be is slightly different.  The important point here is to have a max width of x8.

Back on the AMC720, first reboot it to take the changes into account.  Then you can check that PCIe link is taking into account by the systemoperational (only the last 8 lines are relevant):

Code Block
languagebash
l-utca:~ # lspci
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:1d.0 USB controller: Intel Corporation DH89xxCC USB2 Enhanced Host Controller #1 (rev 08)
00:1f.0 ISA bridge: Intel Corporation DH89xxCC LPC Controller (rev 08)
00:1f.2 SATA controller: Intel Corporation DH89xxCC 4 Port SATA AHCI Controller (rev 08)
00:1f.3 SMBus: Intel Corporation DH89xxCC SMBus Controller (rev 08)
00:1f.6 Signal processing controller: Intel Corporation DH89xxCC Thermal Subsystem (rev 08)
00:1f.7 System peripheral: Intel Corporation DH89xxCC Watchdog Timer (rev 08)
01:00.0 Co-processor: Intel Corporation DH89XXCC Series QAT (rev 21)
01:00.1 Ethernet controller: Intel Corporation DH8900CC Series Gigabit Backplane Network Connection (rev 21)
01:00.2 Ethernet controller: Intel Corporation DH8900CC Series Gigabit Backplane Network Connection (rev 21)
01:00.3 Ethernet controller: Intel Corporation DH8900CC Series Gigabit Network Connection (rev 21)
01:00.4 Ethernet controller: Intel Corporation DH8900CC Series Gigabit Network Connection (rev 21)
02:00.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:00.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:01.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:04.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:05.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:08.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:09.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)
03:0c.0 PCI bridge: PLX Technology, Inc. Device 8764 (rev ab)

...