The European Processor Initiative (EPI), a project with 28 partners from 10 European countries aiming to make the EU achieve independence in high-performance computing (HPC) chip technologies and infrastructure, is proud to present key results achieved in phase one (2018-2021).
These results stem from three major research and innovation domains, the General-Purpose Processor (GPP), Accelerator (EPAC) and Automotive streams, complemented by a number of common activities.
General-Purpose Processor (GPP)
Atos, a global leader in digital transformation, cybersecurity, cloud and HPC, is the lead partner of the General-Purpose Processor (GPP) stream. Together with SiPearl, the company bringing to market the high-performance and low-power European processor, and other EPI partners, they defined the architectural specifications of Rhea, the first generation of the EPI General-Purpose Processor (GPP) implementation and its future derivates.
With 29 RISC-V cores, the Arm Neoverse V1 architecture used by SiPearl to design Rhea will offer an effective, scalable and customisable solution for HPC applications. Architectural decisions were taken following a co-design methodology and by analysing the performance of advanced intellectual property (IP) blocks. A scalable network-on-chip (NoC) to enable high-frequency, high-bandwidth data transfers between cores, accelerators, input / output (IO) and shared memory resources was also optimised by SiPearl.
To allow early software development and performance estimation of the EPI processor while the GPP was still at the implementation stage, a GPP virtual prototype was designed and used.
“We are proud of our success in designing a powerful GPP leveraging cutting edge technologies and IPs built and deployed exclusively by European universities and industrial leaders. We are confident that we will soon demonstrate the instrumental role of this GPP in enabling a European exascale computing machine, the next breakthrough in the HPC domain the world is expecting.” – said Stream Leader Emmanuel Ego (Atos).
“At SiPearl, we are very proud to bring to life the joint project of the European Processor Initiative. We worked hard through close collaboration with the initiative’s 28 partners – the scientific community, supercomputing centres, leading names from industry and innovative start-ups – who are our stakeholders, future users, and clients. With the release of the Rhea processor, we will all contribute to ensure European sovereignty in HPC applications such as personalised medicine, climate modelling, and energy management.” – said Philippe Notton, founder and CEO of SiPearl.
Rhea will integrate technologies by EPI partners and offers unique features in terms of memory architecture, memory bandwidth optimisation, security, and power management.
Memory controllers are one of the most critical IPs when it comes to GPP performance. To help evaluate architectural choices, CEA developed a complete simulation platform with specific instrumentations to analyse controller efficiency in driving the high-bandwidth HBM2E memories. The platform allows efficient analysis of the memory device interface thanks to the decoding and tracking of all memory commands and data. The HBM2E subsystem was simulated with multiple random and directive patterns targeting different traffic shapes and involving all the controller features in maintaining the HBM2E efficiency.
A number of state-of-the-art embedded security features and key technologies were also designed in this stream. These include the standalone Security Management System (SMS) security IP developed by ProvenRun, providing advanced, common-criteria certified, sovereign security IP for HPC and edge processors.
To further bolster security, the University of Pisa contributed a set of crypto IPs, called “Crypto Tile”, integrated in the Rhea GPP by SiPearl. This provides a hardware security module with full security services for high-end symmetric (AES with 9 cipher modes), asymmetric (ECC, ECDSA, ECIES, ECDH) and hashing (SHA2/SHA3) cryptography, delivering several orders of magnitude of increased throughput and decreased energy cost as compared to a software solution.
The Crypto Tile also includes secure key storage and secure IP configuration, side-channel attack protection, on-chip true random number generation (TRNG), support of Linux kernel drivers, extreme key lengths for maximum security levels and high speed en(de)cryption throughput thanks to AXI4-based interface towards DMA and Arm or RISC-V programmable cores. Post-quantum cryptographic support is also provided thanks to real-time implementation of Lattice algorithms such as Crystals Kyber and Dilithium.
Given the importance both of reducing the carbon footprint of future generation computing systems and enabling higher computation capabilities in post-Dennard scaling electronics, energy-efficient computing has been a key consideration in EPI from the outset. For this purpose, an open-source, RISC-V based power controller was designed by University of Bologna and ETH Zurich and integrated into the Rhea processor, harnessing advanced control and artificial intelligence (AI) algorithms for the power management of large-scale systems-on-chip (SoCs).
In addition, based on STMicroelectronics technology on power solutions, Atos and E4 Computer Engineering designed and manufactured the Voltage Regulator and Management reference platform to test the Board Management Controller (mapped inside a field-programmable gate array, or FPGA).
EPI GPP has achieved register transfer level (RTL) completion status as a result of the first phase of the EPI project. The Rhea full design implementation is currently at the validation stage using emulations.
The main result of general-purpose processor activities in EPI phase one, the Rhea processor, will be instrumental for the launch of European exascale supercomputers in 2023.
The EPI accelerator stream set out to deliver energy-efficient acceleration for HPC and AI workloads. With the European Processor Accelerator (EPAC) test chip proof of concept, EPI has demonstrated that it is possible to create an exclusively European design, while the use of open-source instruction-set architectures (ISAs) ensures freedom from proprietary licences and export restrictions.
This stream has fully embraced the open-source philosophy of give and take, contributing to the expansion of the RISC-V ecosystem and adding to the LLVM compiler database. The EPAC systems and FPGA software development vehicles make full use of the Linux operating system and contribute to the community with patches, device drivers, and additional functionality to popular open-source HPC software packages such as OpenMP and MPI. Furthermore, parts of the hardware such as the STX (stencil/tensor accelerator) were developed using a permissively licensed open-source approach around the PULP platform.
“The accelerator stream in EPI has emphatically proven that the RISC-V vector approach has the potential to transform the HPC sector, with designed-in-Europe architectures capable of delivering high performance on a low energy budget,” commented Stream Leader Jesús Labarta (Barcelona Supercomputing Center). “The work also epitomizes European traditions of open science and collaboration. Partners across Europe have joined forces to create something that no single organization could have achieved by itself. By working with open-source technologies and projects, the EPAC stream has helped expand the RISC-V ecosystem, making this technology viable for an increasing number of applications in the future.”
Figure 1 EPAC Test Chip on a test PCB
EPAC set out to provide a proof of concept for European-designed, RISC-V vector architectures for HPC acceleration. A suite of technologies has been developed to this end:
EPAC offers exceptional programmability, with generic codes being run successfully on the test chip with minimal modifications and a software development vehicle to support programmers. It is a genuine example of co-design, with a continuous integration system and rapid application of improvements in response to feedback.
Coordinated by Infineon, a leader in automotive microcontrollers, the Automotive Stream has paved the way towards road-capable autonomous cars, thanks to the proof of concept for an innovative embedded high-performance compute (eHPC) platform and associated software development kit (SDK). This platform, in combination with a downsized, vehicle-tailored, general-purpose processor, meets the increasing demand for computing power in future cars in a cost-efficient, economically viable and functionally safe way.
“Overall, the achievements are evidence of collaboration, synergies and the team spirit which characterised the research work in the automotive stream”, – said Stream Leader Knut Hufeld (Infineon). “With its focus on cost-effective, safe and certified automotive solutions, it can be seen as a driving belt for the overall profitability of European processors in the field of HPC.”
Figure 2: BMW X5 EPI Test Car and the EPI computing system in the rack.
The main achievement was demonstrated in a road-approved BMW X5 car to show the proof of concept for a pioneering eHPC Microcontroller Unit (eHPC MCU) which is integrated in a specially designed flexible modular computing platform (MCP) together with several EPI technology IPs. Numerous test drives were performed to collect data and evaluate test scenarios involving parameters of autonomous driving.
Among other features, the platform includes AI-supported integrated cameras and Elektrobit radar imaging analysis software, with integrated preparation for use of EPI accelerators in the system. It is the result of a close cooperation among the 16 partners in Stream4 aiming to fulfil its objectives of specifying a suitable eHPC Platform, define its architecture and develop the necessary software development kit (SDK).
Infineon also expanded the automotive microcontroller in terms of its architecture and performance ability so that it can act as master and control one or several accelerators. Relevant aspects were safety, security, fall back or redundancy for reduced application, with regard to the top Automotive Safety Integrity Level D (ASIL D) at system level, which is required for autonomous driving applications.
The platform is scalable and open for further technologies. The MCP has various slots for other technologies developed as part of EPI, including:
Test runs reveal that EPI now has specific technologies suitable for autonomous driving up to at least level 4 – where the vehicle drives independently most of the time – thus paving the way for the future.
In addition to the hardware platform, this stream also included the development of a complete software ecosystem, based to a large extent on software products by automotive software specialist Elektrobit. This area also comprises the automotive eHPC platform software stack, including the classic automotive open operating system architecture (AUTOSAR) development for Auto eHPC MCUs, and the adaptive AUTOSAR development for HPC GPPs and the L4Re hypervisor (virtualisation) that are crucial for automotive applications.
With regard to safety, a specific concept was jointly created for a software lockstep, thus contributing to an overall EPI safety concept.
After this three-year initial phase, the results and findings will be continued in further projects.
This stream acted as a provider for other technical streams. Excellent collaboration, both internally and with other streams, helped mitigate issues caused by pandemic-imposed travel restrictions and allowed the stream to meet its objectives, enumerated below.
This stream established a co-design process to shape the design of European processors. Simulations and models with different levels of detail and precision were created to identify the impact of design decisions on the performance of future applications. A benchmark suite of over 40 applications was used to support co-design and later evaluate the EPI processors. Applications have also been prepared to run on future EPI systems, by adapting and testing them on comparable hardware platforms and emulators.
The specification of a “common platform” architecture was defined and used as a backbone for architecture exploration, as the starting point for the GPP implementation, and to define guidelines for security and heterogeneous integration.
Another major result was the integration of the power management design in the GPP specifications: power management firmware, off-chip integration consolidating power distribution board design, PLDA integration, and consolidation of the power management hardware integration.
Work was also done on multiple aspects of the support of system software development: general and hybrid programming environments, OpenMP and MPI runtimes on both GPP and RISC-V sides, OpenMP extra threads support for dynamic load balancing (DLB) and the introspection-based scheduling mechanism in the LLVM OpenMP runtime, offloading for both GCC and LLVM toolchains, testing of power and energy monitoring libraries on available reference ARM Platform, and the resource manager.
Another notable achievement was the development of three tools – gem5, MUSA and SESAM/VPSim – for a complete multi-level simulation environment that provides relevant virtual prototypes for a wide range of needs encountered in the EPI streams. These tools demonstrated broad capabilities, including detailed chiplet- and NoC-level simulation, system simulation for software design, and performance evaluation for design space exploration and hardware co-design activities.
“I was delighted to take over the management of a successful stream, midway through the project, that was able to provide a very high level of co-design, a comprehensive set of benchmarks, and useful simulation platforms that allowed the project to envision processor architecture, effective power management, and to start delivering libraries fit for the new system. I would like to thank Romain Dolbeau, who started the stream, and all the work package leaders and teams for their passion and kind spirit of collaboration.” – said Stream Leader Jean-François Blanc (Atos).
“I’m proud of the outstanding results achieved by EPI teams after only three years of cooperation, paving the way towards Europe’s technological sovereignty. I’m particularly impressed we delivered our objectives on time with a limited budget, despite the unprecedented working conditions due to the terrible COVID-19 pandemic. This has created favourable conditions for the launch of the next phase and its successful delivery of the European processors and accelerators for the EUPEX (EUropean Pilot for Exascale) and TEP (The European Pilot) projects, the precursors to European exascale systems.” – said Eric Monchalin (Atos), chairman of the EPI Board.
The European Processor Initiative (EPI) is a project currently implemented under the first stage of the Framework Partnership Agreement signed by the Consortium with the European Commission (FPA: 800928), whose aim is to design and implement a roadmap for a new family of low-power European processors for extreme scale computing, high-performance Big-Data and a range of emerging applications.
Knut Hufeld, Senior Director R&D with Infineon and an Automotive Stream representative in EPI, talked about the developments in the stream with journalist Ralf Hartmann.
Infineon is known for microcontroller embedded real-time applications, so the European Processor Initiative EPI seems to be an ideal research field?
Definitely yes! With electric and autonomous driving as one of our main future issues, there will be an ever-increasing demand for computing power. Sophisticated driver assistance systems are one thing but new and highly complex applications like environment recognition in real-time will be another. Individual applications may be demanding, but operations like the fusion of sensor data from cameras, radar, or lidar systems are a real challenge. Conventional automotive microcontrollers are not capable to process the mass of information.
How could you bridge this impending gap in computing power?
One approach would be to take reliable automotive microprocessors like Infineon’s AURIX™ which are functional safe up to ASIL-D, and for the algorithms with an intense computing effort you just add processors from the consumer sector to be integrated with little or no adjustment or modification. This way, you create a safety/security configuration with the automotive-qualified real-time microprocessor controlling and monitoring the number crunchers from the consumer world.
Sounds like a makeshift constellation.
Indeed, using consumer parts is nothing but a work-around solution. First, because neither functional safety nor long-term reliability are ensured; second, they are not designed to the system, and moreover, they tend to use way too much energy.
What is the approach in EPI?
Regarding high-performance computing, HPC, the approach is to build a general-purpose processor, a GPP, which is designed for both high-level computing power and low-level energy consumption alike. This is where downsizing comes into play, which means that the number of computation kernels is reduced to what is necessary for automotive. We are talking about a derived variant of the GPP capable to serve typical automotive requirements on a minimum level and hence close the gap.
With its expertise in Automotive and as leader of Stream4 in EPI, which aspects does Infineon have to consider?
In the end, any technical achievement should be economically viable. Considering the high development costs, the challenge is to create downsized variants derived from HPC processors suitable for embedded constellations like in Automotive. Once they meet a high enough demand, they can be produced in sufficiently high unit numbers. The good news, recent developments indicate that the market will move in that direction, soon.
So, you bet on a win-win situation for HPC technology and the embedded sector?
As I said, the prospects are good. With a high demand for downsized variants and their mass production in the embedded sector, HPC can benefit from overall profitability generated by automotive as the driving belt. With this obvious solution, the high investments will pay off and HPC has a chance to become self-sustainable.
…which seems to be the key aspect in Stream4 as the mere automotive part in EPI?
That’s exactly it. Infineon coordinates the automotive part with partners like BMW, Elektrobit, Menta or Kalray, but also academic partners like Karlsruhe Institute of Technology, the Universities of Pisa and of Zagreb and Barcelona Supercomputing Centre. Our achievement is the implementation of a prototype automotive compute platform with integrated accelerators and other main IP blocks. Now, we are waiting for EPI partner SiPearl to come up with its first generation of HPC general-purpose processors. Then we can integrate and test them.
What will be your role in EPI’s upcoming evolutions?
We will still be present, albeit in a different way. In the first round, the Stream4 automotive team managed to accomplish promising results implementing the eHPC automotive platform, which reassures our pursued direction of development. In the upcoming round, we will adapt Infineon’s role towards an Industrial Advisor for embedded computing as an associated partner. This way, the EPI Initiative can take a full share of the limited financial funding resources for HPC technology, while we can shift our focus to setting up complementary projects under the European initiative KDT JU which then may give the opportunity for further synergies within EPI. In the meantime, we will, of course, meet our contractual obligations like exploiting the achievements of EPI. Regarding an effective promotion through face-to-face meetings, we hope that the current restrictions due to the COVID-19 pandemic will soon be lifted.
October started pretty busy for EPI partners, who attended various events presenting the latest from the Initiative.
The calendar was launched by members of Atos and CEA attending the RISC-V Week in Paris, which encompassed two major groups: The 2nd RISC-V Meetings, organized by IRT Nanoelec and CEA and The Scientific Day of IRT SE & GDR SOC2: RISC-V for critical embedded systems, organized by IRT St-Exupéry and GDR SOC2.
Romain Dolbeau from Atos gave a talk titled “European Processor Initiative: challenges & opportunities for RISC-V accelerators in an HPC platform”. He described the currently anticipated architecture of the EPI design and how to leverage this architecture in the software, using open standards. In particular, he touched upon how the EPI project is developing IP for a set of RISC-V-based accelerators designed to connect directly to the processor network-on-chip. He also discussed EPI plans to integrate those IPs in a silicon device, and how other accelerators IP designers could leverage EPI to create new high-performance multi-chip processing devices.
At that same event, Denis Dutoit from CEA gave a talk titled “European Processor Initiative: First step towards a made-in-Europe high-performance microprocessor”. After an introduction on High Performance Computing new challenges and associated technology/architecture evolution, the presentation highlighted the EPI position statement on generic computing, accelerator with RISC-V and design methodology. The presentation concluded with an explanation of EPI’s roadmap towards a wide range of applications from Exascale computing to embedded HPC.
Italian partners also had a very busy week presenting EPI in several workshops and distinguished lectures. Fabrizio Magugliani from E4 Computer Engineering presented at the workshop “Edge computing: Quando il Cloud è troppo lontano”, which confirmed the need for integrating the different streams of data (coming from IoT, the Edge, Industry 4.0 and similar sources) in a consistent view, including the pre-processing of data at the Edge and a HPC-powered analysis in a centralized facility, applying AI and HPDA techniques to the streams and then storing and capitalizing on the ‘knowledge’ extracted from the data. He introduced the roadmap and goals of the EPI and highlighted that the roadmap is fully consistent with the computational need of such a chain of processing.
Only two days later, he also presented at the 12th International Conference on Internet and Distributed Computing Systems in Naples. The speakers listed in the rich agenda detailed how the rapidly evolving technology for Distributed Computing Systems enables to use more computation and ‘intelligence’ at the Edge, and how to maximize the effectiveness of different architectures and components. While these components are ideal for analyzing data at the Edge, more elaborate analysis demands the use of more powerful systems. Having the same Instruction set at the Edge (running Arm devices) as well as in the centralized computing facility is definitely a value, making the EPI processor ideal for seamless integration of the data streams with the EPI-powered centralized facility.
Coming from EPI’s academic wing, University of Pisa’s Prof. Sergio Saponara was invited to give a lecture at the IEEE I&M France Chapter/GDR SoC2 Workshop. He held an IEEE IMS DL about ACES (autonomous, connected, electrified and shared) vehicles and related instruments and measurement/perception tools. The IEEE DL presented the opportunities and challenges of the ACES trends, with a focus on high-performance machine-perception sensors (like Radar, Lidar, Camera), navigation and positioning technologies (Inertial Measurement Units, Global Position/Navigation Satellite Systems). The challenge of the High-Performance Computing, needed on-board the vehicles to process in real-time such large amount of data, using either deterministic signal and data processing techniques or new machine learning and AI tools, have been also discussed. To this aim, the opportunities offered by EPI have been also presented.
Addressing the HPC user community, EPI Chair of the Board, Jean-Marc Denis, attended two HPC User Forums this week – the first in Lugano, Switzerland, and the second in Edinburgh, Scotland, where he presented the road so far, the latest developments and the Initiative future steps.
Across the ocean, another EPI industrial partner, Menta, attended the Arm TechCon in San Jose, California with their exhibition space at which they offered EPI materials for the interested attendees.
The automotive segment of EPI also had a busy week. Karlsruhe Institute of Technology held the first automotive summit in cooperation with the Tongji University in Shanghai. Many representatives from research and companies in China such as BMW, Infineon, Intron and UAES (Bosch in China) participated and gave interesting presentations about their current and future R&D projects. One major topic was autonomous driving and how it can be accomplished with the currently available computing resources, also with solutions ranging from edge and cloud computing to embedded HPC. This discussion naturally included KIT’s participation in EPI, which is why Prof. Jürgen Becker gave an interesting talk about the recent research activities in Germany and Europe also mentioning the on-goings in EPI and the known partners in the consortium. The project and plans were well received by the audience.
October schedule is still not finished, with several more events to go, all of which are announced at EPI’s web calendar: https://www.european-processor-initiative.eu/events/, serving as a perfect ramp-up to Initiative’s attendance at Supercomputing in Denver, US and EFECS conference in Helsinki, Finland in November. Meet us there!
Jean-Marc Denis, HPC International Business Director at Bull, the EPI consortium coordinator, presents EPI at HPC User Forum in Detroit. In the presentation, Jean-Marc focuses on industrial perspectives of the project, the state of the art in HPC industry and our vision to tackle the exascale challenges.
Video is available here