Visualise The World

GDC last week was stiff with VR headsets and other virtual - and augmented – reality equipment, some of which I have reported on already but there was one interesting item I have had to keep under wraps until today’s official launch date.

The PowerVR team at Imagination briefed me on ‘Furian’ which they are calling the first new PowerVR architecture since Rogue was launched in 2010 and there is enough there to make this a very interesting event indeed. Imagination is aiming this new generation explicitly at the 7nm process node with VR, ADAS and notepad PCs in mind.

Where the Rogue architecture was designed for a 28nm process shooting for 60FPS gaming on 2MPixel screens, Furian* is looking for 120FPS on a 4K screen; capable enough to drive a low latency VR setup in a mobile configuration. In support of this, Imagination tells me that they have basically doubled everything: twice the fillrate and twice the SIMD width per cluster, which doesn’t get us to the 8x performance boost implied by the new frame rate and resolution targets but they assure me that with the improved scalability and performance density, that sort of performance is now achievable by adding more clusters within the power limits.

There are a lot of changes just to get the improved graphics performance (more of which elsewhere) but purely from the vision processing point of view, there are some intriguing developments. One of the more interesting is that down in the guts of the cluster where all the compute gets done, Imagination has split out what was a unified cache into separate texture, instruction and data cache memories, allowing them to take advantage of data memory banking to implement a scatter-gather scheme similar to that found on vision-specific DSPs. The rogue architecture was already able to implement lane swizzling and shifting, so this new feature brings Furian into a much more direct feature comparison with those cores.

The base ALU has also changed. Where Rogue has all its instructions going through a single path containing multiple arithmetic units as well as logic and other functions, Furian splits these out into an arithmetic-only path with a separate parallel path for everything else. Imagination does not claim that this path is simultaneous issue, so we’re not looking at some sort of hybrid SIMD/VLIW architecture but given that this secondary path is used for multi-cycle instructions, some degree of parallelism may be available and the simplified structures will certainly lend themselves to higher clock rates and/or lower power operation.

Vision algorithms typically work on fixed point and integer operands so any GPU that plans to compete needs to take that into account. The PowerVR series 7XT GPUs were equipped with separate integer ALUs operating in Vecn mode depending on whether 32, 16 or 8-bit operands were chosen and the launch materials for Furian specify that there is a Vec2 mode for FP16 operands but there is nothing on integer resources. Privately, Imagination has assured me that Furian inherits “some aspects” of series 7XT integer handling but without giving details. That’s a loose end that will need to be tidied up before we can really say how this generation will perform as a vision processor.  

Imagination has been going through some tough times recently. They are still holding on to their prized position as sole GPU supplier to iOS products but that is both a blessing and a curse as the rest of the market gradually slips away to competitors with greater freedom of action. Will the new Furian architecture help them regain their lost market share? With projected headset sales in the single digit millions,  VR is not going to do it for them even if they manage to deploy a ray tracing version but vision functions in mobile and automotive will only get more important and Imagination have traditionally had loyal support from TI, Renesas and Socionext.

According to Imagination, multiple licenses have already been taken and the designs delivered to customers so we should be seeing product announcements within a year. As usual, there is no hint as to who these customers might be but given that this is a high end XT architecture that really doesn’t play in the mid-range handset market of their usual customers, we should expect something from one of those automotive players.

*Yes, the name is something to do with Vin Diesel but for the sake of decency, I shall draw a veil over the mental process that linked Riddick to this GPU

The Khronos Group recently launched an initiative to standardise the way VR applications access the many available hardware platforms  that have arrived on the scene over the last couple of years. On its website Khronos identifies industry demand as driving the initiative and judging by the number of companies that have added their names to the announcement and the industry leaders who have supplied quotes, that is no exaggeration. Khronos supplies a graphic to show the extent of support:

And a selection of quotes will give a flavor of just how enthusiastic the industry is for this standard to come into being.

“With VR on the verge of rapid growth across all of the major platform families, this new Khronos open standards initiative is very timely. We at Epic Games will wholeheartedly contribute to the effort, and we'll adopt and support the resulting API in Unreal Engine,” Tim Sweeney, founder & CEO, Epic Games.

“Khronos’ open APIs have been immensely valuable to the industry, balancing the forces of differentiation and innovation against gratuitous vendor incompatibility. As virtual reality matures and the essential capabilities become clear in practice, a cooperatively developed open standard API is a natural and important milestone. Oculus is happy to contribute to this effort,”  John Carmack, CTO, Oculus VR.

“Open standards which allow developers to more easily create compelling, cross platform experiences will help bring the magic of VR to everyone. We look forward to working with our industry colleagues on this initiative,” Mike Jazayeri, director product management, Google VR.

There are more (and you can see them in full on the announcement page) but just those three cover a huge chunk of the industry, and include three companies that are critically important to the success of this move.

2017 is being billed as a crucial year for the VR industry, with headsets and software at last coming together in a way that might just provide a breakthrough - although in my opinion the jury hasn't even left the room yet. There are rumours of new headsets, new technologies including eye tracking with foveated rendering and optical navigation systems which require new sensor technologies and new software to make use of them so standardisation can only help.   A second graphic from Khronos helps to explain the problem and their proposed solution:

Their basic point is that the market is fragmented wth multiple proprietary runtimes and driver interfaces  and that this impedes the creation of widespread VR experiences that can easily run across multiple platforms. Developers can only afford to support a limited number of platforms so fragmentation leads to less content choice for consumers, and slower VR market growth. Khronos’ proposed standard will include cross-platform APIs for the various sensors, tracking devices, controllers and displays that go into a VR system, solving that problem and stimulating growth of a VR software ecosystem.

This is not the first time that Khronos has moved quickly to head off market fragmentation in the face of rapid innovation; the timely introduction of the Vulkan graphics API last year and the swift move to provide API and file format support for deployment of deep learning based vision systems are two other recent examples of this responsiveness.

This is unusual in a collaborative industry body of this type and is a testament to the organisation and its members. A standard like this can only help, and if the adoption is anything like as widespread as Vulkan, it could turn out to be one of the crucial factors in making this market work.

We've reached a significant milestone here at Visualise The World with the launch of the inaugural issue of The VPU Report!

What is it? It's a direct-to-the-point, detailed analysis of vision processors, as SoC or as IP, from four of the leading companies in the field. Future quarterly issues of the report will present up to the minute information on VPUs available from a different selection of companies building up into a detailed resource for marketing and management professionals. The current issue starts the ball rolling with analysis of Movidius, Intel, Ceva and Inuitive: four of the most interesting companies operating in the consumer edge device category.

At just under fifty pages, packed with over twenty detailed diagrams, plus feature and performance tables along with concise description, the report strips down to basics and presents just the facts so that you can quickly absorb technical details in a convenient, easy to read style.

Along with that, you will find a quick summary of the companies reviewed (useful when so many entrants into this field are startups) and commentary to shed light on the potential and pitfalls of VPU design.

If you are interested or involved in the coming revolution in vision enabled devices, or if you need to know about the chips and IP that will run the vision and neural network algorithms that will power them, you will need to get this report. And occasionally there will be a scoop, such as in this issue a first public description of an as-yet unreleased device!

The report actually speaks for itself, so I've attached a pdf of the contents and introduction. Take a look, then click on the link above and I'll get in touch to give you rates and subscription options.

the_vpu_report_q4_2016_toc.pdf

Machine learning is an umbrella term for a group of related technologies which are critical to the development of visually-aware systems. There has been a huge explosion of interest in these technologies driven by their astonishing successes and a consequent explosion in the range and diversity of tools, formats and platforms available for developers to work with.

Neural Networks in particular are being successful at pattern matching and classification tasks and their new popularity is shown by the number of incompatible development and deployment tools that are emerging. Now The Khronos™ Group (www.khronos.org) is moving to reduce confusion and enable interoperability by announcing two new standardisation initiatives in that area.

A news release from Khronos today states that a new working group has been set up to define an “… API independent standard file format for exchanging deep learning data between training systems and inference engines.” and “… the OpenVX™ working group has released an extension to enable Convolutional Neural Network topologies to be represented as OpenVX graphs and mixed with traditional vision functions.” (The full press release is available here).

The first move will establish a standard way to move defined networks from the training phase, which is typically done offline using a wide range of rapidly developing tools and techniques, and the inference phase where the network is actually used in an application. This phase, especially in embedded systems is usually mapped onto highly optimised and constrained platforms so by providing a standard exchange format, Khronos’ Neural Network Exchange Format (NNEF) will eliminate much of the need for development and deployment platforms to share implementation details.

By concentrating on a flexible format to exchange data, rather than on the structure of the actual networks being deployed, Khronos aims to avoid stifling innovation in a fast-moving technology while still achieving the objective of reducing deployment friction. According to Khronos President Neil Trevett: “So many companies are actively developing mobile and embedded processor architectures the market is in danger of fragmenting, creating barriers for developers seeking to configure and accelerate inferencing engines across multiple platforms.”

With the related move by the OpenVX working group to provide an API to a set of standardised neural network topologies as well as, crucially, to allow the import of non-standard nets developed elsewhere into the openVX graph structure, Khronos is again smoothing the path for deployment of machine learning without restricting innovation.

This move by Khronos is very timely, with many companies looking for a viable deployment path into end-user equipment while at the same time, researchers in both academia and industry are continuing to come up with novel ideas almost on a weekly basis.

Much of the impetus for the new working group came from Adasworks, one of the frontrunners in promoting machine learning software. Laszlo Kishonti, Adasworks’ CEO, commented that: “We see the growing need for platform-independent neural network-based software solutions in the autonomous driving space. We cooperate closely with chip companies to help them build low-power, high-performance neural network hardware and believe firmly that an industry standard, which works across multiple platforms, will be beneficial for the whole market.”

It is unusual to see a standards body get ahead of the market in this fashion and it is to the credit of Khronos that they are doing it is a way that is sensitive to the needs of the technology. The new working group is only just starting up and there is no word on when a ratified standard will emerge but based on the recent activity from other Khronos groups, which has been very rapid for a body like this one, perhaps we will see something within the next year.