Close
0%
0%

InterNoC

A Network-on-Chip for the System-of-Systems Era
Enabling mixed interface communication between embedded devices (MCU, sensors etc.)

Similar projects worth following
Nowadays, many embedded systems host a significant number of micro-controllers and processors (i.e. IoT devices, vehicles, airplanes, satellites) and as this number continues to increase, traditional bus solutions will start to fail on those platforms as well. Network-on-Chip (NoC) not only offer a scalable solution for MPSoC interconnects but they can also provide a uniform platform of communication to embedded systems with multiple interconnected devices.

This project aims to build an open-source FPGA-based NoC where devices can be connected to each other, transparently, enabling mixed protocol communication between multiple master / slave devices, without the need for a shared bus and at the same time allowing each device to operate in its own clock domain.


The InterNoC

To eliminate any confusion the following terminologies will be set and used throught the project:

  • (AXI) Master / Slave : corresponds to the internal interconnect structure of the FPGA
  • Master / Slave device : corresponds to the connected to the device Master/Slave from a system perspective.
  • Master / Slave interface: corresponds to the interface used by a master / slave device.

The Architecture

The InterNoC architecture tries to address the N-masters to M-slaves mixed protocol communication issues throughout the physical and application layer by:

  • Implementing an abstraction layer between peripheral interfaces and on-chip interconnection accesses.
  • On-chip interconnection is managed in the form of AXI memory-mapped master/slave scheme. 
  • Conversion IP cores from a peripheral interface to an AXI-MM master/slave
  • In the prototype level the on-chip interconnection is managed by a combination of
    • AXI interconnect for master-slave accesses
    • Shared-memory via use of AXI Mutex for master-master acceses

The Communication

From the user perspective, InterNoC acts like a mixed protocol communication hub of: N(masters) x M(slaves).

  • Communicating through via the NoC will be done in an addressable scheme. Each Master interface port will have a pre-defined address. In order for a user component to communicate to another the Master component will send a packet over the communication link containing the address of the destination component followed by the data. This scheme aims to make the NoC transparent to the user. Master devices will communicate with other connected devices as if they accessed a register address.
  • Interfaces will be fixed in the development of the prototype system, i.e. fixed number and placement of SPI, UART, I2C interfaces. Nevertheless, it is in future plans to extend the capabilities discussed and create a fully dynamic, partially re-configurable, mixed communication platform.

The Hardware

For the current phase of the project the hardware will be comprised of a bare bones artix 7 FPGA implementation along with a set of 2.54mm headers for interfacing common buses (SPI,I2C,UART). 

In the future and once we have achieved to have a dynamic platform from the IP side, the solution can be scaled to various applications from low end FPGA solutions with common buses to more advanced automotive CAN bus interfaces to high end PCIe and DDR interfaces. 

In the end the ulitmate goal would be to enable a user to inerface an external device with a range of devices from simple I2C sensors to DDR3 memories through the same InterNoC solution.

The Concept

InterNoC_Board.pdf

Proto board layout outputs

Adobe Portable Document Format - 409.16 kB - 10/07/2017 at 13:57

Preview
Download

InterNoC.pdf

Proto board schematic

application/pdf - 451.74 kB - 09/26/2017 at 15:36

Preview
Download

  • 1 × XC7A35T-1FTG256C Xilinx Artix 7 FPGA
  • 1 × S25FL128SAGBHIA00 Memory ICs / FLASH Memory
  • 1 × APTD2012LSURCK LED/Red
  • 1 × APTD2012LCGCK LED/Green
  • 2 × TPS7A9101DSKT LDO Voltage Regulator

View all 39 components

  • Multi-Slave Write Access Demo

    Lefteris Kyriakakis12/12/2017 at 13:32 0 comments

    After a few weeks of days of debugging and a few weeks of life getting in the way a first real-case scenario demo of the InterNoC functionality is presented here...

    Setup

    This log presents a demo of the InterNoC hardware design (see Fig. 1) implemented on a CMOD Artix-7 development board.

    Simplified InterNoC Demo Hardware RTL
    Fig. 1 Simplified InterNoC demo hardware RTL 

    In the demo, a master device (STM32 dev. board) is programmed to perform write accesses of different byte count through a UART interface. Two devices were connected to the InterNoC and acted as SPI slave devices:

    1. An STM32 dev. board
    2. An oscilloscope with interface decoding capabilities

    These devices were chosen in order to facilitate the verification of the InterNoC correct operation. Furthermore, as a means of in-circuit verification the Xilinx Integrated Logic Analyzer (ILA) was used.

    The following was defined in-order to communicate using the InterNoC using two mockup messages "ok" and "test" sent respectively to slave 1 and slave 2:

    #define INTERNOC_WRITE_CMD 0x0
    #define INTERNOC_READ_CMD 0x1
    
    #define INTERNOC_SLV_0 0x0
    #define INTERNOC_SLV_1 0x1
    
    // bytecount | access type | slave address in InterNoC
    #define INTERNOC_CONSTRUCT_HEAD(byteCount, cmd, addr) ((byteCount << 5) | (cmd << 4) | (addr))
    
    uint8_t write_addr_0 = (uint8_t) INTERNOC_CONSTRUCT_HEAD(0x2, INTERNOC_WRITE_CMD, INTERNOC_SLV_0); 
    uint8_t write_addr_1 = (uint8_t) INTERNOC_CONSTRUCT_HEAD(0x3, INTERNOC_WRITE_CMD, INTERNOC_SLV_0); 
    
    char buffToSend[2] = "ok"; // TX buffer for slave 1
    char buffToSend1[5] = "test"; // TX buffer for slave 2

    Demo

    The master STM32 dev. board was programmed to perform the following loop:

    • Write four bytes to slave device in address 0x0
    • Wait 2500ms
    • Write to two bytes to slave device in address 0x1
    • Wait 2500ms

    The oscilloscope received correctly the transmitted 4 bytes as presented in the scope capture (see Fig. 2):

    Slave 2 scope decoder receives message correctly
    Fig. 2 Slave 2 scope decoder receives message correctly

    Following up is a video of the presented setup running the demo:

  • Prototype Board Design#2

    Chris Gkiokas10/07/2017 at 14:42 0 comments

    In this log, the first renders of the pcb layout are presented and a major change in the design.

    First off, the change was that we changed the FPGA package as obviously the 484 pin was an overkill for this design. The package was switched for the 256 pin BGA that shaves off a little bit form the chip cost and saves us the unnecessary layers that the 484 would require to escape the BGA.

    In the picture above we can see the functions of the board. The user would simply wire up the master and slave interfaces required and provide a 5V supply either by plugpack or the two dedicated pins next to the jack input. The device then will handle the communication between these devices regardless of the protocol used by each one.  

    The top LEDs and two buttons have mainly debugging purposes, while they could be informative for the user they do not provide any function for the main purpose of the board. As for the configuration of the board a JTAG header is used in standard SPI flash mode, in the future a usb interface is planned to provide the user a more streamlined interface for the dynamic reconfiguration of the FPGA.

    Evidently the board could be made a lot denser if the 2.54mm headers in this layout would allow, however for this prototype we kept to the original plan of a 10x10 cm board.


    The board uses a 6 layer stackup where:
    • Top is mixed power & signals
    • Ground layer
    • Signal layer
    • Signal Layer
    • Ground layer
    • Power layer

    The layout was pretty straight forward as there are no particularly high-speed signals or differential pairs for our interfaces. Probably the most intense part of the layout was getting the BGA fanout and getting the power planes to reach the FPGA as all 3 power rails use the bottom layer to get there. For that reason, the bypass capacitors of the FPGA were placed on the bottom as placing them on top would result in a bigger loop.

    There is a lot of work still to be done before going on with manufacturing, for once the design and layout must be quadruple checked, the silkscreen markings and the general aesthetic presentation of the board must be designed and at least some spice simulations, mainly on the SPI clock signal traces must run to verify the signal quality.

    The new schematic pdf file along with a pdf of the layout plots will be uploaded here, and of course the whole project can be found on GitHub.

  • Mixed Protocol Interconnect Dev#2

    Lefteris Kyriakakis10/06/2017 at 18:05 0 comments

    CURRENT DEV STATUS:

    In this development log the mixed protocol interconnect functionality is demonstrated in simulation.

    First, a custom AXI SPI component was developed that enables the control of an SPI Master. The development of this component was necessary in order to simplify the overall design without the use of the Xilinx SPI IP as well as to integrate, inside the custom component, functions necessary for the correct packet communication. interface through the AXI bus protocol.

    The developed component's RTL architecture is presented below:


    The component can be configured through the following parameters:


    At the current development stage there AXI addresses are ignored. Any incoming AXI write requests are carried out as SPI Master transmissions and any AXI read requests are carried out as SPI Master receptions. In the future addresses could be used to dynamically specify the CLK divider value as well as the CPOL / CPHA configuration of the SPI.

    The replacement of the Xilinx SPI component changed the overall demo design into the following architecture:


    Finally the full functionality of the developed design was demonstrated in a developed test-bench. The test-bench drives the UART_RX_0 port as if a master device (i.e. Arduino) was connected to the interface UART_RX_0/UART_TX_0 and was transmitting a 40-bit mockup packet with 32-bit data ("0x44332211") at a constant rate with destination, a slave device connected to each of the axi_spi_master_x SPI interfaces. Each time the transmission was complete a new packet was sent to a different slave device.
     
    The demo system achieved a peak throughput of 9.2 kBps. The interfaces where configured at:

    • UART Baud-rate: 115200
    • SPI sclk: 6.25 MHz

    The figure below is exported from Vivado 2016.4 and presents the simulation results of the demo test-bench system.


    In the figure above the time it takes for a 40-bit packet carrying 32-bit values to be successfully transmitted from a UART master device to an SPI slave device is marked between the yellow/blue markers and 434.72 us.

    Furthermore, as in the figure it is clearly demonstrated the concept of the mixed protocol interconnect since a UART master device can address and transmit data successfully to each SPI slave device connected to the respective interfaces.

    COMING UP NEXT.... 

    • Develop Master - to - Master communication scheme and present simulation results.
    • Develop IP component to provide JTAG access for configuring different parameters of the  master/slave interfaces, i.e. baud-rate, spi clock divider etc.

  • Prototype Board Design#1

    Chris Gkiokas09/26/2017 at 15:10 0 comments

    In this first board design log the schematic design for the prototype is presented along with a BOM that will be linked in the project page.

    The proto board will be used to support and verify the IP development and stand as a proof-of-concept for this project. The board will be designed using KiCad EDA and will be fully open under the CERN Open Hardware License: GitHub Page

    The board will feature:

    • A Xilinx Artix 7 FPGA(XC7A35)
    • 24 common interfaces in total.
      • Master: 3 SPI, 4 I2C & 1 UART
      • Slave: 6 SPI, 8 I2C & 2 UART
    • 24 LEDs for debugging (one for each interface)
    • 2 GP Buttons mainly for test resets


    Overall the design is quite minimal and it follows mainly the artix7 reference designs with the interfaces in attached straight on the FPGA I/O banks. The board requires an external 5V power supply that can be provided either by a common DC Jack or 2 dedicated pins.

    In all reference designs, of course a set of switching regulators is used to provide for the power rails, in our solution a set of LDO regulators was used to reduce BOM cost. The design specs followed closely those of the reference design providing two 1A capable rails of 1V & 1.8V for core and aux supply respectively and a 2A 3.3V rail for the I/O banks of the FPGA. As the TI TPS7A regulators have pgood and enable function the timing sequence was kept to spec using RC Delay to keep the ramp ups separated at approximately 10ms, inherently monotonic due to LDO nature.


    Sequence simulation on Texas Instruments Tina-TI:


    As in the reference designs the clock source is a 100MHz Abracon mems oscillator. The board also has 24 LEDs and two tactile switches to aid prototyping.


    For the interfaces, while we thought about using level translation on the interfaces to increase the robustness of the board, the BOM cost with that many is skyrocketing so a simple Zener- resistor protection was used, the interfaces are 5V logic tolerant due to Zener action. The number of interfaces was decided purely upon how many 2.54mm pitch headers can fit on the edge of a 10x10cm square. The assignment of master and slave interfaces is not hardware decided of course and can be interchanged from the fpga, for that reason there are no external pullups.

    Master Interfaces:


    Slave Interfaces:


    The main chip will be a Xilinx Artix 7 FPGA (XC7A35T-1FGG484C) and will be configured by JTAG with SPI flash configuration.


    The BOM is linked here, however resistor values on the interface channels may change once the final trace impedances on the PCB are calculated, especially in the SPI interfaces where the clock speeds can theoretically be over 20MHz. 

  • Mixed Protocol Interconnect Dev#1

    Lefteris Kyriakakis09/26/2017 at 13:00 0 comments

    INTRO:

    The FPGA Hardware code is managed in the GitHub repository: https://github.com/egk696/InterNoC under the license: GNU General Public License v3.0

    CURRENT DEV STATUS:

    This is the first dev. log for the InterNoC RTL project. In this log the idea for a mixed protocol is developed and presented on a demo design in Vivado 2016.4. To eliminate any confusion the following terminologies will be used:

    • (AXI) Master / Slave : corresponds to the internal interconnect structure of the FPGA
    • Master / Slave device : corresponds to the connected to the device Master/Slave from a system perspective.
    • Master / Slave interface: corresponds to the interface used by a master / slave device.

    For flexibility and a fixed protocol the decision to use AXILite Master/Slave Bus interfaces was taken. This allows the use of native Xilinx IP cores as well as a unified bus scheme for the FPGA interconnect.

    To begin with, the mixed protocol interconnect needed a converter that accepts packets from a master interface and converts it to AXILite master transactions. This component was first developed and is presented below:



    The Master-Slave interconnect is based on the Xilinx AXI Interconnect. This choice was made due to its native and easy to implement structure as well as its scalability and flexibility, i.e. it manage multiple clock domains, protocol conversions, arbitration policies and much more.

    The final design is build of 3 master interfaces (2 UART and 1 JTAG) all connected to 4 slave interfaces (SPI). For the purposes of prototyping the SPI used is a Xilinx IP Core (https://www.xilinx.com/support/documentation/ip_documentation/axi_quad_spi/v3_2/pg153-axi-quad-spi.pdf) while the UART interface is a personally modified version of the UART components presented in (https://www.nandland.com/vhdl/modules/module-uart-serial-port-rs232.html).

    For the current design the baud rate for the UART master interfaces is set at 115200 and SPI slave devices are operating at 25 MHz. In the future solutions will be investigated to support a user configurable baud-rate and clocking frequencies for the interfaces.

    Accessing slaves is done via an embedded in the FPGA address map:


    For a master device (i.e. Arduino) to access a slave device, a 40-bit packet must be sent, over the master device's interface (i.e. UART), that follows the following format:

    Control Bits [39:39]
    Control Bits [39:37]
    Address Bits [36:32]
    Data Bits [31:0]
    Indicates R/W access:
    "0"=> Write
    "1" => Read
    Indicates Tx/Rx Byte Count:
    "00" => 1 byte
    "01"  => 2 bytes
    "10"  => 3 bytes
    "11"   => 4 bytes
    Support for up to 32 devices
    i.e. 0x0, 0x1, 0x2, ..., 0x1F
    Arbitrary data for slave device


    COMING UP NEXT....

    • Master - to - Master communication scheme
    • Simulation and Debugging of the demo design

View all 5 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates