Application Note
NG-ULTRA
bitstream loading security
Ver 1.0.0
Jan 2022
Table of Content
Release Notes
Document History
Revision | Date | Editor | Modification |
1.0.0 | 04/02/2022 | PN | Initial Document |
Related documentation
Reference | Description | Version |
RD1 | NG-ULTRA_BringUp_UserGuideV1.3 | V1.3 |
RD2 | NG_ULTRA_BitstreamLoadingSecurity_v1.0.0 package | V1.0.0 |
RD3 | NxBase2_v2.5.1-UL1 | V2.5.1 |
Acronyms
Acronyms | Description |
ASW | Application Software |
BL0 | Boot Loader stage 0 |
BL1 | Boot Loader stage 1 |
BSM | Bitstream Manager |
CMIC | Configuration Memory Integrity Check |
CTR | Counter mode for block encryption |
DMA | Direct Memory Access |
DSP | Digital Signal Processor |
eROM | embedded ROM |
OTP | One Time Programmable (memory) |
ROM | Read Only Memory |
SoC | System on Chip |
SPI | Serial Peripheral Interface |
SPW | SpaceWire |
TCM | Tightly Coupled Memory |
TMR | Triplar Modular Redundancy |
Summary
This application note, the reference designs and the associated scripts are intended to show NG-ULTRA FPGA bitstream loading security process.
Introduction
The NG-ULTRA component is composed of a large embedded FPGA and a digital processor subsystem based on ARM Cortex R52 cores. The bitstream manager (BSM) manages the FPGA fabric configuration.
Fig 1 – NG-ULTRA bloc diagram
NG-ULTRA FPGA bitstream security is managed through the implementation of encryption mechanisms applied to the data to be protected. Security function that encodes a digital data using a secret encryption key, making the encoded data unintelligible unless it is decoded using a decryption key.
Test environment
Tools
Hardware:
NG-ULTRA Bring-up board
Software:
Nxmap
Nxbase2
Security loader test package tools reference designs, scripts and binary files
Install plugins NXBASE2 full options
Please install the plugins full options to enable the loading of BTS file with NxBase2.
Copy the directory full_options into the two directories:
nxbase2_cli/plugins
nxbase2_cli/nxbase2/plugins
Bring-up board set up
Bypass mode
Mode useful to test and debug, to bypass all security process.
Name | Jumper | Comment |
USER3 | J33 | Bypass Loader Security |
USER4 | J35 | Bypass Mrepair |
Tab 1 – Bypass mode selection
Note:
This mode can be control through Nxbase2 software (overwrite jumper setup)
This mode is only available on NG-ULTRA V1
JTAG interface
JTAG interface is chained from the SOC to the FPGA and allow to debug the SoC code or to load a bitstream in the FPGA. On the bring-up board, the NG-ULTRA JTAG can be connected by the FPGA SPARTAN6 with the Nxbase2 commands or by the debug mictor interface with the lauterbach trace 32/debug probe or openOCD.
Name | Jumper | Value | Comment |
JTAG selection | J46 | 0 | Mictor interface is master of JTAG |
1 | Nxbase2 is master of JTAG |
Tab 2 – JTAG master selection
Note:
This mode can be control through Nxbase2 software (overwrite jumper setup)
JTAG clock must be < configuration clock / 2
NG-ULTRA configuration mode
Name | Jumper | Value | Comment |
MODE1 MODE0 | J43 J42 | 0 0 | Mode Normal 0 FPGA: JTAG SOC: SPW, SPI Flash, UART |
MODE1 MODE0 | J43 J42 | 0 1 | Mode Normal 1 FPGA: SL_PAR_16, JTAG SOC: SPW, SPI Flash, UART |
MODE1 MODE0 | J43 J42 | 1 0 | Mode FPGA only FPGA: SL_PAR_16, JTAG |
MODE1 MODE0 | J43 J42 | 1 1 | Mode Test |
Tab 3 – Configuration mode selection
Note:
0 = jumper unset, 1 = jumper set
This mode can be control through Nxbase2 software (overwrite jumper setup)
FLASH SPI configuration
Name | Jumper | Value | Comment |
MREPAIR_SEQ_Bypass | J38 | position 1-2 | No SOC Mrepair bypass |
position 2-3 | SOC Mrepair bypass | ||
BOOT_SRC | J37 | position 1-2 | boot from SPI Flash memory |
position 2-3 | boot from Spacewire | ||
Flash_Mode | J34 | position 1-2 | Flash mode sequential |
position 2-3 | Flash mode TMR | ||
J47 | 0 1 | Internal Flash selected External Flash selected (J41) | |
Flash_Num[0] | J32 | position 1-2 | Flash_Num[0] = 0 |
position 2-3 | Flash_Num[0] = 1 | ||
Flash_Num[1] | J36 | position 1-2 | Flash_Num[1] = 0 |
position 2-3 | Flash_Num[1] = 1 |
Tab 4 – FLASH SPI configuration selection
Note:
Internal flash selected, Flash_Num selects Flash 0, 1, 2 and 3 on board
External flash selected, Flash_Num selects Flash 0, 1, 2 on board and Flash 3 external
This mode can be control through Nxbase2 software (overwrite jumper setup)
Configuration clock
Configuration clock source | Jumper (exclusive) | Frequency |
NG-ULTRA Internal clock | J18 | 100 MHz ~10% |
Bring-up board U12 OSC clock | J20 | 50 MHz |
External SMA clock J21 | J19 | - |
Spartan Slave Parallel 16 clock | J25 | 12 MHz |
Tab 5 – Configuration clock selection
Note: When programming the Security OTP, the user should set the J20 to have the frequency BSM clock required to write into the OTP memory. (40MHz < clk BSM < 80 MHz)
SOC clock
SOC clock source | NG-ULTRA Pin | Frequency |
Bring-up board U6 OSC clock | REF_CLK1 | 25 MHz |
External SMA clock J6 | REF_CLK2 | 24-280 MHz |
Tab 6 – SOC clock selection
User clock
User clock source | Name | NG-ULTRA Pin | Frequency |
Bring-up board U7 OSC clock | GCLK_TOP_L | IO_B10D09_P_CLK | 25 MHz |
Bring-up board U7 OSC clock | GCLK_TOP_R | IO_B11D09_P_CLK | 25 MHz |
Bring-up board U7 OSC clock | GCLK_BOT_L | IO_B05D08_P_CLK | 25 MHz |
Bring-up board U7 OSC clock | GCLK_BOT_R | IO_B03D08_P_CLK | 25 MHz |
External SMA clock J9 | SMA_CLK_IN | IO_B05D09_P_CLK | - |
Programmable Ref. clock by Nxbase2 | CLKangie | IO_B12D09_P_CLK | - |
Programmable Ref. clock by Nxbase2 | CLKangie2 (Only available on bring-up board V2) | IO_B03D09_P_CLK | - |
Tab 7 – User clock selection
Bring-up board LED
Name | Domain | LED | Comment |
CFG READY | CONFIG | D12 | Configuration done without errors |
CFG ERROR | CONFIG | D13 | Configuration error |
CFG TRIGGER | CONFIG | D14 | Configuration done with 1st errors |
VDD_CORE | VOLTAGE | D15 | Voltage CORE |
3V3 | VOLTAGE | D16 | Voltage 3V3 |
1V8 | VOLTAGE | D17 | Voltage 1V8 |
HSSL VDD CORE | VOLTAGE | D18 | Voltage HSSL CORE |
SOC VDD CORE | VOLTAGE | D19 | Voltage SOC CORE |
12V | VOLTAGE | D21 | Voltage 12V |
DDR3 V | VOLTAGE | D35 | Voltage DDR3 |
BOOT ERROR | SOC BOOT | D31 | Boot loader error |
BL0 RUN | SOC BOOT | D32 | BL0 running |
Tab 8 – Bring-up board LED
Note: At power ON, all voltage LEDs have to be lighted.
Bring-up board user GPIOs
Name | Direction | NG-ULTRA Pin | Comment |
Usw1 | Input | IO_B07_D00 | Switch 1 |
Usw2 | Input | IO_B07_D01 | Switch 2 |
Usw3 | Input | IO_B07_D02 | Switch 3 |
Usw4 | Input | IO_B07_D03 | Switch 4 |
Usw5 | Input | IO_B07_D04 | Switch 5 |
Usw6 | Input | IO_B07_D05 | Switch 6 |
Usw7 | Input | IO_B07_D06 | Switch 7 |
Usw8 | Input | IO_B07_D07 | Switch 8 |
PushB1 | Input | IO_B07_D08 | Push button 1 |
PushB2 | Input | IO_B07_D09 | Push button 2 |
PushB3 | Input | IO_B07_D10 | Push button 3 |
PushB4 | Input | IO_B07_D11 | Push button 4 |
PushB5 | Input | IO_B07_D12 | Push button 5 |
PushB6 | Input | IO_B07_D13 | Push button 6 |
PushB7 | Input | IO_B07_D14 | Push button 7 |
PushB8 | Input | IO_B07_D15 | Push button 8 |
Uled1 | Output | IO_B07_D16 | Led 1 |
Uled2 | Output | IO_B07_D17 | Led 2 |
Uled3 | Output | IO_B07_D18 | Led 3 |
Uled4 | Output | IO_B07_D19 | Led 4 |
Uled5 | Output | IO_B07_D20 | Led 5 |
Uled6 | Output | IO_B07_D21 | Led 6 |
Uled7 | Output | IO_B07_D22 | Led 7 |
Uled8 | Output | IO_B07_D23 | Led 8 |
User_IO1 | Bidirectional | IO_B06_D00 | User IO 1 |
User_IO2 | Bidirectional | IO_B06_D01 | User IO 2 |
User_IO3 | Bidirectional | IO_B06_D02 | User IO 3 |
User_IO4 | Bidirectional | IO_B06_D03 | User IO 4 |
User_IO5 | Bidirectional | IO_B06_D04 | User IO 5 |
User_IO6 | Bidirectional | IO_B06_D05 | User IO 6 |
User_IO7 | Bidirectional | IO_B06_D06 | User IO 7 |
User_IO8 | Bidirectional | IO_B06_D07 | User IO 8 |
User_IO9 | Bidirectional | IO_B06_D08 | User IO 9 |
User_IO10 | Bidirectional | IO_B06_D09 | User IO 10 |
User_IO11 | Bidirectional | IO_B06_D10 | User IO 11 |
User_IO12 | Bidirectional | IO_B06_D11 | User IO 12 |
User_IO13 | Bidirectional | IO_B06_D12 | User IO 13 |
User_IO14 | Bidirectional | IO_B06_D13 | User IO 14 |
User_IO15 | Bidirectional | IO_B06_D14 | User IO 15 |
User_IO16 | Bidirectional | IO_B06_D15 | User IO 16 |
User_IO17 | Bidirectional | IO_B06_D16 | User IO 17 |
User_IO18 | Bidirectional | IO_B06_D17 | User IO 18 |
User_IO19 | Bidirectional | IO_B06_D18 | User IO 19 |
Tab 9 – Bring-up board user GPIOs
Note: User GPIOs are activated low
Bitstream download process
To configure the fabric, the BSM uses rows of drivers. Each driver row manages a subset of logical rows. Each driver row is managed by a Loader module. This Loader module includes all FSM to access to configuration memory.
A bitstream is a collection of Frames. Each frame as a header and a body. Frame header is a 32b word protected by EDAC. A bitstream includes 2 CRC, Frame CRC that checks the integrity of each fabric configuration frame and Bitstream CRC that checks the integrity of the whole bitstream.
Fig 2 – BSM bloc diagram
The BSM is composed of several I/O controllers:
JTAG interface with dedicated IOs
PAREXT (16b parallel) interface with dedicated IOs
PARUSR(16b parallel) interface connected to the Fabric
AHB slave interface connected to the SOC
Each controller is connected to 2 engines bloc to access the loaders:
Frame engine is used to fabric configuration to manage bitstream format. It’s frame access
Direct engine is used to debug and test to manage address, data format, not encapsulated in a bitstream. It’s direct access
The security module is in charge of:
Accessing OTP memory that contains all security information and identification value.
Controlling accesses depending on lifecycle
Decrypting the bitstream
Checking the bitstream authentication
Test_001 is an example of loading a non-encrypted bitstream with security mode enabled and disabled.
Loader OTP memory
The loader OTP stores security related data such as keys, seeds for key generation, identifier, anti-rollback counter, secure mode configuration.
Name | Size | Mask | Protection | Lock | Description |
KMACBTS | 128 | Yes | ECC | No | Key used to authenticate the Bitstream |
KENCBTS | 128 | Yes | ECC | No | Key used to decrypt the Bitstream |
KFRKAES2 | 64 | Yes | ECC | No | Fresh Re-Keying Master Key 2 |
KFRKAES1 | 64 | Yes | ECC | No | Fresh Re-Keying Master Key 1 |
KFRKMAC2 | 64 | Yes | ECC | No | Fresh Re-Keying Master Key 2 |
KFRKMAC1 | 64 | Yes | ECC | No | Fresh Re-Keying Master Key 1 |
MSEED | 32 | No | ECC | No | Key Masking Seed |
LFC-CTR | 160 | No | ECC | No | Lifecycle counter |
FEATURES | 32 | No | ECC | No | run_max_error_cnt (8b) fresh-rekeying enable (8b) |
TEST_MODE_DISABLE | 32 | No | ECC | No | Disable test mode (3b, LSB) |
RLB_CTR | 128 | No | Redundancy | No | Anti-rollback counter |
ICID | 32 | No | Redundancy | Yes | Unique Integrated Circuit ID |
IDCODE | 32 | No | Redundancy | Yes | JTAG IDCODE |
Tab 10 – Loader OTP memory contains
Note:
Before accessing the OTP memory, it will be bisted and the OTP supplementary address 0x3 and 0x4 must be written with value 0x5555 and 0xAAAA respectively
When programming the Security OTP, the user should be careful to have the main configuration clock, CLK_BSM between 40 and 80 MHz
The user should be careful to program the OTP ECC-protected fields only once. If an ECC6 protected field is programmed multiple times, there’s a great chance that the ECC signature becomes unreadable and ends up ‘bricking’ the chip by generating a DED error upon booting
Some data are masked with the key masking seed before writing into the OTP memory
Test_002 bist the loader OTP memory.
Test_003 program the supplementary address to get the write access to the OTP and program the OTP identifier as IDCODE and ICID.
Test_004 dump the OTP memory.
Lifecycles
The lifecycle value is stored in the OTP memory. It determines the bitstream configuration interface and the type of BSM accesses authorized and the type of bitstream (crypted or encrypted).
Fig 3 – Lifecycle diagram
Note: In order to write a lifecycle in the OTP, the user should make sure that the OTP user code, ICID are already written.
Lifecycle interfaces and accesses
Cyphered | JTAG IF | PAREXT IF | AHB IF | PARUSR IF | |||||||||
Frame Write | Direct Read | Direct Write | Frame Write | Direct Read | Direct Write | Frame Write | Direct Read | Direct Write | Frame Write | Direct Read | Direct Write | ||
MANUFACTURER | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
BRINGUP | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
LABO_DEV | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
LABO_SECURE | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
LABO_SPACE | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
PROD_DEV | No | Yes | Yes | No | Yes | Yes | No | Yes | Yes | No | Yes | Yes | Yes |
PROD_SECURE | Yes | No | No | No | No | No | No | Yes | No | No | Yes | Yes | Yes |
PROD_SPACE | Yes | No | No | No | No | No | No | Yes | No | No | Yes | Yes | Yes |
EOL | NA | No | No | No | No | No | No | No | No | No | No | No | No |
Tab 3 – Lifecycle interfaces
Note: The direct access is proceeded by direct engine with BTS file and the frame access is proceeded by frame engine with BBS or NXB file.
Anti-Rollback
A cyphered bitstream contains the rollback counter. Deciphering engine compares it to the anti-rollback counter stored into the OTP memory. The check algorithm depends of the life-cycle.
LIFECYCLE STEP | Test Mode Enable (SCAN) | Rollback Counter Check |
MANUFACTURER | Yes | Equal |
BRINGUP | Test_mode_disable control | Equal |
LABO_DEV | Test_mode_disable control | Equal |
LABO_SECURE | Test_mode_disable control | Equal |
LABO_SPACE | Test_mode_disable control | Greater than |
PROD_DEV | Test_mode_disable control | Equal |
PROD_SECURE | Test_mode_disable control | Equal |
PROD_SPACE | Test_mode_disable control | Greater than |
EOL | Test_mode_disable control | No accesses |
Tab 12 – Lifecycle anti-rollback check
Test_005 program the security data, lifecycle data, encryption keys and anti-rollback counter.
Bitstream encryption
Depending on the lifecycle, the encryption must be used to properly load a bitstream.
Bitstream encryption step:
Nxmap has to generate a bitstream in BBS format file
Remove the comments into the .BBS file to simplify the following padding step
Some informations should be added at the beginning of the .BBS file
Rollback counter (128b)
TAG (128'h2e90_0db0_07ab_7eac_ce55_c0de_de7e_c7ed)
Size of the bitstream + padding words expressed in number of 32b word (128b)
Bitstream payload
Padding words
Run the encryption script Utils/Encryption/bitstream/encrypth.sh
input_folder=VALUE | Read plaintext bitstream in choosen folder (default plaintext) |
output=VALUE | Write cyphered bitstream in choosen file (default $plaintext) |
output_ext=VALUE | Write cyphered bitstream with choosen file extension (default txt) |
output_folder=VALUE | Write cyphered bitstream in choosen folder (default cyphertexts) |
nonce=VALUE | Use VALUE as nonce (default 0) |
key_enc=VALUE | Use VALUE as key_enc (default 0) |
key_mac=VALUE | Use VALUE as key_mac (default 0) |
define=VALUE | Compile Encryption C sources with -d VALUE option (combinable) |
wrong_mac | Replace MAC computed value by 0 array |
Note:
Cyphered bitstream is constituted of 128b frames whereas uncyphered bitstream contains 32b frames. To match size, uncyphered bitstream must be padded to 128b with dummy data inserted after the uncyphered bitstream
There are 0, 1, 2, or 3 padding word maximum
Example:
Bitstream BBS file without comments:
12345678
01e061da
00000001
6001988d
cacababe
e0001c28
94f59cd2
Bitstream BBS file modified before encryption:
0000000000000000000000000000000 -> ROLLBACK COUNTER
2e900db007ab7eacce55c0dede7ec7ed -> TAG
00000000000000000000000000000008 -> Size of the bitstream + Padding words
12345678 (7*32b words + 1 padding=8*32b words)
01e061da
00000001
6001988d
cacababe
e0001c28
94f59cd2
00000000 -> padding word
Encryption script command:
./encrypt.sh –input_folder=. --key_enc=7625412a7625412a7625412a7625412a
--key_mac=7625412a7625412a7625412a7625412a
Encrypted bitstream BBS file:
12345678
00000000
00000000
00000000
feedb0a5
ec7edd07
1abe15e1
2efacade
00000000
00000000
00000000
00000000
00000000
00000000
00000000
00000000
6ceecd01
1b31513e
9b6b2a3a
dc803c91
2b1af62b
c97aedcd
5cc3dbf3
ae9e5210
bdd846d9
ff9f1d65
c5446ab4
b2cd4d09
2f5e5e5d
5b01e618
8bae17f8
fb36b169
79780431
ea48e29e
4f64fd7e
f682c447
90e7e237
613e643c
8b27c27c
b8560fc4
Test_006 is an example of loading a crypted bitstream.