the xfund blog

XMOS recently published a white paper entitled A Programmable Revolution, setting out some differences between the XMOS and traditional FPGA approaches, in an attempt to win the hearts and minds of FPGA developers. It has followed that up with a couple more whitepapers called XC for Verilog Designers and Unified Design for Hardware and Software.

Problem

In the long run, XMOS could offer a replacement to many multi-technology systems involving FPGAs and microcontrollers. However, XMOS does not offer in a drop-in replacement for either. The problems with switching to XMOS from existing FPGA/FPGA+microcontroller or even other pure MCU systems are on several different fronts – technological and commercial. For example, the XMOS chip:

1. Will not run the same software as the MCUs out of the box – it will require quite a bit of modification
2. Clearly requires quite a bit of PCB design reworking – not only because of the new chip but also because of the new peripherals and some of the separation of PHYs that were previously integrated into the MCU packages
3. Often requires a different skillset to develop using it – FPGA designers cannot be expected to just pick up and XMOS chip and start using it in a commercial project, even with the help of the whitepapers!
4. Requires replacing or retraining a team of FPGA developers with a team of embedded software engineers to use XMOS chips. This is complicated and expensive. There is no “whole product” offering including (re)training courses and certification.

Solution

The XMOS idea of “replacing” low-end FPGAs and MCUs in the marketing and sales pitches is, I think, a bit misguided. Whilst it may be a great long-term vision, XMOS should focus on bridging the gap and making it easier for people to use the technology. I think XMOS should focus on how it can “talk to everything”, and how it can be a complementary technology, rather than a competitive technology.

How do you get an existing team of FPGA developers to start using XMOS?

Position it as a complementary technology, position it alongside other products, and gradually take over the development and production platforms from the inside out. The second whitepaper – giving a few code examples – is a very early and very tentative step towards this. But without a physical platform enabling developers to explore the benefits and costs of using both platforms side-by-side, you won’t persuade people to give up what they know so well and what their entire skillset and product is based on.

This means XMOS building simple demo/development boards that (for example) provide a high-speed interface between a low-end FPGA and an XMOS chip and getting these into the hands of the FPGA developers. Designers should be encouraged to experiment with mixed FPGA-XMOS platforms, using each device for what it is good for (both technologically and in terms of development effort and speed). XMOS-FPGA boards offer compelling development platforms, whilst also easing the technology transition.

Anyone want one?

I recently read a great article by Andrew Richards, CEO of Codeplay, which really highlighted some important points in the next phase of computer architecture development.

Know Your Market

The first thing it highlighted was the absolute necessity to know your market.

You can skip the games bits if you like, but at the end of the day, Intel’s paying for Larrabee with future sales to games players and visualization clients. So, no games, R&D bills don’t get paid: no Larrabee.

Andrew points out that despite there being a seeming mantra in non-game-developers that game graphics engines display data-parallelism, this is in fact nonsense, and that the parallelism they actually display is far more complex. Intel however had approached the GPU design with this data-parallel view, and because of this belief, a cache-coherent design was implemented. Read the article for more information – but basically, cache coherency algorithms do not scale unless data accesses are almost totally independent (ie data parallel). You can see now where the architectural decisions were influenced by not knowing the market well enough…

Understand Software!

The second big point here that I find particularly interesting is that using cache-coherent multi-threaded programming systems is hard. I hope Andrew doesn’t mind me quoting quite a large chunk which highlights this:

But, in fact, this style of multi-threaded programming is very hard, and instead game developers like using task-parallel systems with strictly-defined inputs and outputs, and asynchronous memory copies, because that is actually easier to program than traditional multithreaded programming [at least in games.] So, instead, other GPUs use systems of separate ["local"] memories. This means all inter-core communication has to be explicit, but it also means programmers have control over it.

Why is this particularly interesting? Well, because generations of computer scientists have been arguing for explicit concurrency, local memories and explicit communication between computational elements for decades – for ease of programming, code safety (and formal reasoning), timing guarantees, scalability and the ability to compose code with predictable outcomes. And look, Andrew goes on…

The biggest shock for Larrabee was the discovery that it takes longer to implement something in software on a multi-core CPU than custom hardware.

But why is this a shock? When one looks at the application and the customers, one sees a graphics pipeline with many complex stages that each exhibits substantially different behaviour. A single processor architecture therefore cannot easily cope with this. Some stages might exhibit strong data-parallel behaviour requiring minimal, well-defined local computation and a scatter-gather approach whereas some other stages might require concurrency at a far more coarse-grained level with larger blocks of inter-communication sequential processes.

That means that if you are trying to design a scalable architecture which can handle this in software, you need to create a very general-purpose architecture which can handle both coarse-grained and fine-grained parallelism (as well as middle-grained!), but therefore cannot possibly provide optimal performance for everything (or even, for anything!).

The truth is that if you build this single-unit plus communications bus “scalable architecture”, you’re going to get killed by people building custom hardware, which is carefully optimized for the application. The main reason for not building custom hardware is that it is expensive and time-consuming. But in the graphics world, the software is complex enough that it takes at least as much effort as the hardware, the market is big enough – and willing to pay enough – for the expense of developing that custom hardware.

Dead in the Water?

I don’t actually think that the software-programmed high-performance GPU is totally dead in the water, but I don’t believe it can be done using a single computational element and a communications network. I think you need to design a mixed-granularity architecture with some computational elements using long instruction-word formats that are optimised for graphics processing and others using a more general-purpose instruction set with support for coarse-grained concurrency. And I think one needs to forget entirely about cache coherency…

Lessons Learned? What about XMOS!?

Given the focus of this blog previously on the embedded market and in particular on the XMOS range of processors, I should draw some comparisons. The logic Intel adopted seems, at first, to be fairly similar to the logic of XMOS. Yes – Intel believed that it could overcome the obvious performance deficit of a software-controlled GPU versus custom hardware by offering other benefits – one is assuming these are time-to-market, re-usable portable code, the ability to rapidly adapt to changing specifications in the game engines.

We’re not a million miles away from the XMOS proposition here. What’s the core difference? XMOS is about the embedded market – the belief that a software flow will reduce time-to-market, specification vulnerability, and encourage code re-use and modularity, whilst delivering competitive performance and price.

Well, here’s some key differences. The point of XMOS is that it ought to be able to trounce custom hardware on price. Why? Because the unit volumes sum up over the total market for the entire XMOS sales. Which means that all customers get the benefit of the sales volume for all other customers. This is totally not the case for the GPU market – there is a single market and (virtually) a single application.

Another: code complexity. In the GPU we have highly-complex, performance-critical and large software. In the embedded space, we have mostly “glue” software – code that is used to tie together a couple of different interfaces, code to convert one format to another. Yes, there are performance critical pieces of software, but these are mostly timing-critical hardware interfaces. These can be carefully optimised once to meet the timing specification of that particular interface (take, e.g. USB 2.0) and then re-used by everyone else. There is no need to keep pushing this interface 5% faster, because the rest of the world only talks at the 100% speed (if that!).

Marginal Performance Scalability

This raises the critical point about the marginal performance scalability of the market. All markets constantly demand increased performance. But the question is always what form this takes. In some markets, this means that we must run the same software 5% faster. A great example here is operating systems on PCs. We don’t want the underlying software to change much – but we do want it to run faster.

At the other end of the scale you want to be able to run 5% more software, at the same speed. This is the embedded market – yes, your phone does voice calls, SMS, now we’d like that CPU to handle the camera too. Now the display. Now Bluetooth. Now USB 2.0.

Between the two extremes there is a continuum – but the GPU market is very much at the “operating system” end. This end is the end where we have few options but to throw increased clock speed, better caching, or basically, better custom hardware at the problem. The embedded market is in stark contrast. One doesn’t have to run existing software faster, one just needs to be able to run more software. And this can be solved very easily – by simply adding more “threads”, more “cores”, more concurrent processing resources.

Summary

Basically this comes back to the first point: know your market. The embedded market is ripe for consolidation under the banner of a general-purpose CPU implementing a wide range of hardware interfaces in software. The GPU market is not.

I exchanged a series of emails with Stephen Williams, my local MP. Stephen is the Liberal Democrat spokesman for Innovation, Universities and Skills.

It irritated me that he did not attend the Digital Economy vote, so I made it clear to him how I felt. I received some very strange arguments in reply – and a week ago, sent him the following – currently unanswered.

Dear Stephen,

This email has now been copied into my family – also residents of your ward – and have shown great interest in this series of emails. Please respect our email privacy and use these addresses solely for correspondence on this issue.

I quote from previous emails you sent to me:

“The vote was essentialy symbolic to show my party’s opposition to the Bill as it stood.  We knew we would lose as the Labour and Conservative front benches had already agreed with each other.”

“Because of the “wash up” process, I had no reason to be in Westminster on Wednesday, and so I was in Bristol West meeting constituents.  If I had returned to London at the last minute to vote, then the Bill would have had 48 rather than 47 opponents. But with both Labour and Conservative front benches giving their support to the Bill this would have been a futile journey.”

The Liberal Democrats win seats because voters choose to vote for them – even in the knowledge that they will not win the election.

They choose to vote for them because they take their right to vote seriously and wish to be heard.

You are undermining this principled position by explaining above why you no longer believe in it, or why you believe it does not apply to you.

We know the chances of the Liberal Democrats winning the 2010 election are at best slim (though congratulations to Nick Clegg for the debate), but if we wish to vote Liberal, we still turn out to have our voices heard. The party would currently not exist without the sentiment of those of us who believe in putting what is right above what is obvious.

I would like to understand how someone who appears to share such passion about parliamentary reform can have become so cynical about the processes preventing it. It is absolutely essential that our politicians do not forget the youthful, enthusiastic naivety with which they likely entered politics and allow it to be crushed by process, bureaucracy and experience.

It is only a willingness to believe that anything is possible that will inspire generational social and political change. I would like to believe that this belief – not its crushed skeletal remains – lies at the backbone of the Liberal party.

I think my email speaks for itself. I simply cannot understand this kind of argument coming from a Liberal Democrat. I hope I get a considered reply.

This recession has got us a bit stuck in the UK.

Big companies and chains dominate our way of life – perhaps not to the extent they do in the US, but probably more than any country outside of the US.

And let’s remember, the US is a hotbed of innovation and entrepreneurship. The US has Google, Apple, Facebook. The US has Silicon Valley. The US breeds the desire to change the world into its citizens.

The UK does not. Instead in the UK, high achievers, entrepreneurs and anyone who dares to be different is often ridiculed and mocked whilst they try and resented if and when they succeed. See an interesting post by Ewan MacLeod for more on this.

Reports

Sir James Dyson has recently put out a report – commissioned by the Conservative party – identifying mechanisms by which we can try to reinvigorate innovation in this country in an attempt to reconstruct our economy. The Royal Society also reported recently that we need to invest more in basic science and technology education. Hermann Hauser is shortly releasing a report commissioned by Peter Mandelson – which judging by a recent Financial Times article is likely to spell out more investment in educational and entrepreneurial institutions too.

The really, really bad news – from an economic point of view – is that these guys are right. Why is this bad news? Because the lesser-read paragraphs of the Royal Society report identify that this is a multi-decade return on investment and that there will be no short-term value derived from this investment. Investing in basic science will not – short term – make the UK more globally competitive. Encouraging entrepreneurship will not – short term – fix the generational attitude (described by Ewan) that persists today.

It’s bad news because it doesn’t provide any short-term solution to our current mess. It indicates we are so far off the right track we need to spend decades putting it right…

So – if the UK is educationally and culturally incapable of creating global competition in the short-term through free market forces, are there any other ways to stimulate our economy?

Solution

Well here’s a totally mad solution. Do I believe anyone has the courage to implement it? No. But I’m going to suggest it anyway. And I promise you, it’s far less mad than printing money.

We have far too many large, incumbent, (multi-)national chains which control (and stifle) much of our industry and economy. For the most part, business sectors are run by oligopolies and cartel behaviour is common and well-documented.

I suggest we simply break them up. Force them to separate into a whole host of smaller, competitive companies. Force them to survive without restrictive trade practices and collusive arrangements. This is the best way we currently have on the table to inspire innovation in businesses and in our economy. The market can’t generate it because of a lack of cultural and educational basis. It is time for the government to step in. When free markets can’t deliver – it is the role of the government to fix the economy.

A great place to start is the financial sector – but why stop there? Insurance, utilities, retail, transport, brewery chains, coffee shop chains, air travel, freight…

We need to stop this desperate attempt to reinflate a burst bubble with hot air. Let’s get on with doing something real instead.

I was sitting on another late, filthy train last night and it reminded me of an idea I had a while ago.

How many are furious about the appalling state of our public services in the UK – particularly those that have been privatized? I suspect it is a lot. I suspect there are people who would love to do something about it, but don’t feel empowered with any mechanism to do so.

Well I had a mad idea the other day. I’m going to apply it – conceptually – to the railway network. It could apply to others.

Start a not for profit company that sells train tickets – a bit like TheTrainline (but obviously, not for profit). These ticket reselling companies *make money* on each ticket sold – I’m not sure how much it is (someone look it up?) but it’s at least 2% of the net ticket price. Now, that adds up to a *lot of money* in its own right. We will also sell “membership” for £5 a year. Non-members will have the option to pay an extra £1 on the price of every ticket the buy. And we will accept additional donations.

But we’re not for profit, so how are we going to use that cash, and why do we need it? Well, we’re going to attempt to buy shares in the train operating companies. Controlling shares.

Clearly, our own structure then needs to be examined. A not for profit entity that controls one or more train operating companies needs clear, democratic governance by its members. Paid-up members will have the right, perhaps annually, to vote in new management committees. The management committees will exert controlling influence over the train operating companies – forcing them into line with passenger interests. Furthermore, any dividends paid out on train operating company shares will be reinvested in further train operating company acquisition.

This scheme, given sufficient public momentum, could get to the point where our customers – those who use the trains every day – would effectively own the railway network. This is exactly as it should be. And in a rather odd kind of way, this not for profit co-operative organisation now fills the role a government should with respect to the railway network. We have created a form of distributed microgovernance over public industries.

Do you think the mood is right to do something like this? I’m well aware the hurdles. It’s a critical mass kind of idea. Are we – as a population – unhappy enough to want to make a change?

Just a thought.

i recently critiqued an article by fred destin for its failure to address structural incentive problems with vc firms/funds. he rightly pulled me up and asked me to explain what i meant.

very briefly (since my blog was intended to focus on tech!) – creating the right vc investment vehicle is about creating a structure that aligns the incentives – and therefore the behaviour – of the venture investors, the venture firm, and the entrepreneurs.

percentage-based fund management fees break this model, because they incentivize the vc firm to raise larger funds than are necessary, and to minimize the size of the fund’s management team to maximize the fee return. this does not align with the firm’s investors or the entrepreneurs in which the firm is intended to invest.

then the size of the fund determines the investment upper – and more importantly lower – bounds. the lower bounds are currently set way too high. these bounds should not be set though by the venture firm – as these bounds are what determines the risk-reward tradeoff for the firm’s investor. these upper and lower bounds should instead be set by the venture investor, because it is their capital being used and therefore they who should choose the level of risk-reward.

furthermore, large funds managed by small teams means increased due diligence, more question-asking, more research (which all sound great from a “risk-management” perspective but actually incentivize herding in order to cover the vc’s asses. there are many, many examples of venture firm herding (where they all chase the same industry or company) in recent years – clean energy, nanotech, biotech etc. herding bloats the industry and reduces the returns possible by overvaluing weak investment propositions.

large funds mean that failure is not an option: if you only make a small number of large investments, you frequently cannot afford to let them fail. so you continue to back them way past their natural shelf-life. why do you get away with this? because timescales for funds are too generous – you get ten years of failure before “anyone calls bullshit” (as a vc put it to me recently). this is because the processes involved in vc decision making are too opaque for their own investors and for the companies they back.

finally, the 20% cut of “any profits made” is madness, when you look at the fees the investor is drawing. their late-stage, bloated, herding, ass-covering investments are unlikely to deliver more than a few % return on average – so why should they bother even going after this 20%. their fees cover them anyway.

recap: timescales are too long, investment process too opaque, fees cannot be percentage based (fixed?), incentive for success must be higher, firms must be larger (which would be fine with lower fixed fees – also known as salaries).

the structure of vc firms needs to shift radically, not the behaviour of the individuals involved. asking the current vcs to fix their industry is like asking rbs to fix the banks.

i explained in an earlier post what the rationale is for a chip that can bridge the market (and technology) problems facing the consumer electronics industry today. recent comments by altera have put some interesting context on this.

recap

to recap briefly – ever-increasing demand for product differentiation, shorter product lifecycles, and device mobility has led to demands for flexibility, low-power and low-cost.

application-specific (custom) chips aren’t economical these days – they takes tens of $ms to develop and several years, and market volumes and product lifecycles simply won’t support their development. if custom chips are to thrive, they must serve large, long-term markets… but then they aren’t custom!

more fragmented markets, more products, and short lifecycles demand programmability as a solution. this is where fpgas tended to fit. but even fpgas demand long design cycles. more worrying though, they also use a lot of power, making them unsuitable for many embedded, portable applications or devices that must run on batteries.

altera’s claim

where has altera claimed its niche? well, altera claims that its niche is in high performance consumer electronic applications, such as streaming video over mobile networks – watching youtube on your phone is given as an example. why? well – because it’s only in high-end consumer electronic gadgets that the customer accepts that watching movies on youtube on their mobile won’t be good for their battery life!

in most other embedded or portable applications, power consumption is simply too important to ignore.

altera has justified why custom chips no longer work, and has carved itself out a market which cannot possibly sustain its business model. fast, more power-efficient processors with the capability to deliver fpga-like performance are just around the corner. the first generation of xmos processors already threaten many conventional fpga markets – and along with open-source platforms such as arduino are capturing a significant proportion of the “prototyping/development” space. altera is an immense company – with an immense investment in field application engineers and direct sales channels. it needs more than niche markets to sustain itself, or it will die.

wider context

in almost every other embedded product, general-purpose processors will capture the market: they will be more power-efficient, and quicker to develop with – with a pure software flow.

this logic is almost certainly behind the recent arm/xilinx partnership. arm brings to xilinx the capability to compete with the general-purpose processors and microcontrollers that are taking over the fpga prototyping market, and gives them the capability to compete in production devices with less power-overhead (as the percentage of the product offering that is “gate-array” is much lower), and a quicker time to market (software running on the arm versus verilog/vhdl synthesized on the fpga).

the article is entitled: “How the decades-old battle between ASICs and FPGAs is shifting.” i don’t think it’s shifting. both custom asics and fpgas are disappearing in the rear view mirror.

over the past few months we have seen several criticisms of the venture capital industry from both the outside and the inside. fred destin is notable for explaining fund overcapitalization – the presence of “too much money” – in his article in the telegraph

i give fred a lot of credit for publicly enunciating elements of this problem – it was brave, if a little too posthumous. if you can’t be bothered to read it, here is the rough argument. venture capital is an attractive industry because it promises the allure of potentially huge returns. so more money was poured into the industry. because of a lack of experienced vcs, this inevitably led to increased fund size – the “experience” was a valuable and constrained commodity.

increased fund sizes with a small number of vcs meant larger investment “deal size” – because funds had such a lot of money and so few people the only way they could do their research properly and control their overheads was if they focused on a small number of larger investments.

larger investments are by their nature later stage. because they are later stage, the returns are generally lower (because the risk/disruptive potential is lower) and the quality of the investment opportunity is lower (because companies at a later stage asking for a lot of money are probably more capital-intensive and less efficient).

this justification for resounding failure is compelling because it sounds nice and simple and most of all logical.

logical like why it wasn’t the individual bankers who caused this recession, but the structural incentive mechanism. logical like “we must pay bonuses of $50m to retain and attract top talent” (they just had a bad year). logical like printing money.

the reason i want to take this argument apart is because it blames structural problems for the selfishness, greed, ineptitude and ultimate failure of individuals.

put yourself in the position of a vc. you know how to turn small funds investing in early-stage companies into big successes. everyone has seen your success, so now everyone wants to stick money into your new fund. what do you do?

if you are a good businessman, you recognize what value you delivered and how. you work out a successful, repeatable model. you know what the optimal investment structure is, how much you should invest, at what stage, and in how many companies. and you take the money you need from the best of those who are offering it to you. and you succeed by delivering value to everyone.

but that isn’t what happened. instead, the venture capitalists’ eyes were too big for their stomachs. they saw people waving banknotes at them and blind greed took over. they raised funds that were ten or a hundred times as big as their previous ones. they worked out the bare minimum of management and legal structure possible to manage the cash. and once they had the cash, they set about working out how they personally would make money out of it – not how they would deliver value to their investors, nor how they should deliver value to their investments. they forgot (or chose to ignore) why they had succeeded, and what they were good at – and therefore failed the critical acid test of business: delivering long-term value.

one could forgive a little greed, perhaps? but here’s the problem: funds have been failing for a long time. and rather than doing anything about it, the greedy venture dinosaurs have continued to sit at the top of the feeding chain, swallowing tens of millions in “fees” for managing funds that were losing money through poor investments in the wrong companies at the wrong stage. those unlucky enough to get funded by the old guard found themselves squeezed at every step for the last per cent of the equity whilst those unlucky enough to put money into the venture funds found themselves losing money hand over fist on their investments, paying for sloppy management, poor decisions, and no returns.

but what could they do? i could write a long list: if you acknowledge the fund is too big, break it up. if you acknowledge your deal size is too large, make smaller investments. admit your mistakes to your investors, and make money for them with a smaller pot. hire more people, run more seed funds, train more people – the problem, it was argued, was bandwidth. focus on specific industries, develop expertise (don’t pretend to have it!). funds have had a decade to change tack. instead it is only now, when the failure is public and visible, that they are finding excuses in the industry structure. this, to me, is pathetic.

the venture capital industry as a whole has spent the last ten years suffocating its investments, squandering its investors’ money, and starving out early-stage, exciting, world-changing, “high-risk” ventures by investing in conventional, broken, late-stage, inefficient, bloated antiques.

how many vc funds can demonstrate a half-decent ten-year return? for those than can’t, how many can explain what actions they took to remedy the fund’s failure?

it’s not all bad, there are one or two exceptions. but for the most part, this industry has failed everyone it has engaged with in the last ten years. even now, the question is “who will fix it?” the vcs have become so comfortable on the multi-million dollar rewards for failure that they are waiting, much like the banks, for someone to destroy their industry from the outside.

they have had ten years to build their castle, but the siege has now well and truly begun.

had a brief conversation recently about the sharpe ratio, investment and venture/seed capital. the sharpe ratio, in effect, provides a measure of the risk premium (or excess return) per unit of risk – ie how good is your investment. the higher your sharpe ratio, the better your investment.

angel/seed investment does not give a good sharpe ratio. the risk is relatively high, and the potential return is not enough to justify the risk. later, lower risk investments give a better sharpe ratio, even though the expected return on investment is also lower.

i want to ask two key questions, and provide some initial thoughts.

firstly, why is angel investment still so prevalent if there are quantitatively “better” investments? well, there are two reasons that immediately spring to mind. angel/seed investment requires relatively small amounts of cash. that is cash that individuals can afford to lose. and therefore the assessment of risk is lower. whatever your perception of absolute material risk, the angel’s perception of risk is much lower, because it doesn’t matter if they lose the cash. this ties in with the second reason: angel investing is fun, and therefore delivers value to the investor. most angels will never make money, but they will have an immensely fun time losing it.

however, this argument might also support the theory that seed fund investing is also not worth doing on a “per-deal” basis. to a certain extent, this is true. if you are seeding a company that will need to go through a few seed rounds, then series a, b, c etc and then finally reach an exit, the reward is likely to be small after all of those rounds of dilution, and the risk of the company tripping up along the way is high. this is why seed investments in technology ventures are not looked on favourably by conventional investors.

turning this around though, when the seed capital is the only – or one of very few rounds of funding – the return on investment grows. when the risk can be managed and kept low because of in-house expertise, one finds the per-investment sharpe ratio grows substantially. a great example of where this has happened is the explosion in web (2.0) startups: it doesn’t cost much to get them to profitability, and if you’ve been in the business, you can spot a winning concept. this gives you the high-return, low-risk investment that gives a great investment proposition.

coming onto my second point, it is interesting to see what happens to the sharpe ratio when you consider it as it applies to a series or collection of investments, rather than individual seed investments. the net effect of the formula is to reduce the risk faster than you reduce the expected return, which increases the sharpe ratio.

couple these two together: if you can find a way to get new technology companies to market relatively quickly and cheaply, and minimize the risk by cultivating in-house expertise, and invest in a series of companies, you create a fantastic investment vehicle…