Common mistakes when offshoring Embedded Software Development
INTRODUCTION: Outsourcing refers to the process of handing over a business function or part of a business function/process to a third party. The onset of the digital era has made it possible to seamlessly perform business functions across different countries. Most fortune 500 companies have taken advantage of cheaper labor costs in developing countries and have moved business functions/processes offshore either by outsourcing to third parties or by establishing offshore development centers. The words ‘offshoring’ and outsourcing are used interchangeably.
One of the early candidates to move offshore was embedded software development. Intel was a pioneer when it opened an offshore center in Haifa, Israel, way back in 1974[1]. This was followed by industry giants Texas Instruments[2] (Bangalore, India, 1985) and Motorola (Russia, 1997)[3]. Some of the early efforts at offshoring embedded software development were complete disasters. Companies tried to move peripheral application software development with vaguely defined specifications resulting in software full of errors and cost overruns greatly exceeding anticipated savings. Unlike IT applications which are loosely coupled with hardware and have intuitive requirements, Embedded Software applications are written exclusively for specific hardware platforms and requirements are dictated by the system and environment in which they are used. Regular project management practices usually yield little or no benefit when trying to move Embedded Software development offshore. Coupled with unexpected problems relating to cultural differences with the offshore center, companies have managed to create a unique set of worst practices as described below.
1) Outsourcing too little or incomplete modules of software
Nearly 97% of companies surveyed by the Offshoring Research Institute[4] stated cost as a reason for offshoring, yet the size of deals seems to be shrinking and specialized players are getting smaller contracts.[5]The problem with smaller contracts or trying to offshore few software modules is that additional management at the client’s location can very well exceed any savings. An advantage is that risk is significantly reduced by having smaller contracts besides the benefit of expertise offered by smaller niche companies.
Embedded software though is a different creature. Each software module is intricately linked to the rest of the system; even peripheral application software has strong linkages to the environment the system operates in. It is therefore unlikely that offshoring smaller pieces of software will yield substantial savings. A good approach to estimate savings is by using software estimation and project control tools like SEER-SEM (by Galorath)[6]. Such tools give the user an option of selecting inputs like Size (Number of lines of code), Environment, Complexity etc. and give an estimate of the Cost and Effort involved in the Project. As Embedded Systems have greater complexity and Environment factors, using such tools tells us that cost per module is high for a fewer number of modules but decreases as number of modules increase.
[Figure Deleted]
(Inputs and Outputs of SEER-SEM, Project Management Tool
(Copyright: GA-SEER Technologies))
2) Trying to define a specification in great detail
The specification is usually intended as a first step in software development after specifying the system as a whole. Companies in their eagerness to cut risk or minimize errors, when offshoring Software Development, think that this as a great way to proceed. This approach fails miserably with Embedded Software Development. To examine a better approach we need to see what is inside a Software Functional Specification. A typical software functional specification needs to contain the following sections[7]
A) Introduction
B) Overall Description
C) System Features
D) External Interface Requirements
E) Other non functional requirement (regulatory requirements)
F) Other requirements
Using this template to Embedded Systems brings out some interesting observations. Section D and E (External Interface and Non Functional Requirements) are typically much more detailed and extensive when compared to IT applications as they have information about underlying hardware. Non functional features (features relating to safety, security and regulations) have a large impact on System Features. In fact most System Features in Embedded Software originate from the interaction between External Requirements and Non Functional Requirements. Also it is impossible to predict all System Features without computer simulations. Yet most companies, new to offshoring try to write a specification in great detail, resulting in several project management problems as well as cost overruns. This approach of trying to define a specification in great detail leads to witch hunting (pinning the blame on an individual) and slowing down learning, according to Pawel Brodzinski[8]. We can therefore conclude that this approach not only leads to poor results but also undesirable organizational behavior.
A better alternative is to use an Operational Specification in the first stage of design[9]. An Operational Specification is the user’s view of the system without going into details of its working. Sharing this view of the system with the offshore center is a vital first step towards better understanding and communication. Other important ‘to-do’s’ are having a prototype and imparting sufficient knowledge of hardware. Both these points are discussed further
3) Failing to tutor by prototype and relying mostly on formal training
Several job training methods are available to prepare an offshore center. Most companies prefer to minimize training expense as it increases cost and reduces the incentive to offshore. Many offshore suppliers claim to have ‘trained staff’ ready to take on projects at a moments notice. The cold reality however is that some form of training is unavoidable when moving work to a different location. Many company specific technologies and protocols are completely unfamiliar to people with several years of experience in Industry. So it makes sense to look at various training options and choose the best (cost effective) option. Some common training methods include[10]:
a) Lecture
b) Demonstration
c) Seminar
d) Conference
e) Panel
f) Simulation
g) Self-Discovery
i) On-The-Job
Without going into details of these methods, the most popular methods in the Embedded Systems industry are Lecture/Seminar and On-The-Job training. Industry conferences and Trade shows provide a good introduction but the bulk of learning must be On-The-Job. Nearly 80% of learning takes place on the job and by looking at how others are performing their duties. Yet this vital component is missing in an offshore center. Among other options available to replace this vital component, Self-Discovery seems the best bet. Most offshore locations have people with reasonable engineering skills and the ability to look at a prototype and understand it. If a prototype of the Embedded System is not available, any previous project or Embedded Controller/Processor software provides a useful starting point. Looking at a previous project and its software output tells a lot more than Formal Lecture based training, which seems to be the current preferred option for offshore centers.
4) Imparting little or no knowledge of hardware/system
Embedded Software essentially consists of two components. A device driver component which has instructions to run the microprocessor and the application software which has most of the functionality. Both components interact with either the Hardware or the System. The device driver component has traditionally been handled by the manufacturer of the microprocessor or a closely linked supplier. The reason for this trend is best captured by this statement from microcontroller.com:[11]
“Developing device drivers for a highly integrated microcontroller can be daunting, partly due to the sheer complexity of the device, but also due to some other difficulties”
Trying to offshore device driver development needs in depth knowledge of the processor and several conventions and processes peculiar to each manufacturer. It goes without saying that knowledge of hardware (microcontroller/processor) is a basic requirement.
Developing application software for Embedded Systems needs familiarity with the system/environment. Trying to develop software for controlling an Engine without knowing how an Engine works is very difficult and likely to result in several errors. Despite knowing fully well the importance of familiarizing the offshore center with system/hardware, most companies attempt to offshore both device driver and application software development with a detailed specification in the hope that adhering to the specification will result in code free of errors. We have seen previously why this approach is bound to fail.
Another aspect often ignored while offshoring is the lack of intuitiveness in Embedded Software. For example in IT applications one can guess expected behavior when a button named ‘close’ is pressed due to the fact that behavior is obvious and somewhat standard across a variety of applications. However it is difficult to predict requirements for a temperature sensor failure in an Engine Control Unit (A good example of an embedded system) of a car. Even people with several years of experience in the automotive industry need to think carefully before deciding on the Algorithm. The offshore center usually gets blamed for incorrectly developing software but in reality the problem lies in a misunderstanding of requirements, which are usually not intuitive in Embedded Systems. This is one of the major reasons for lack of satisfaction from an offshore provider. It scores higher than culture/language barriers which are often stated as the major obstacle to offshoring.
5) Trying to impose processes and standards without taking local conditions into account
Most major corporations have software development standards and processes. Several have followed the V-model for software development shown below:
V-Model for Software Development
[figure deleted]
Most job descriptions and functions closely follow this model. For example there is typically one person performing requirements analysis, another performing Architecture Design and yet another performing Unit Testing. When work is moved to an offshore location, this model as well as job descriptions and functions are adhered to scrupulously. It is assumed that people will be found and ready to work in one of descriptions suggested by the model. If employees are not satisfied with their job functions, they are moved around but seldom is the model/job description tinkered with.
The advent of modern tools for software development like autocode generators and model based design tools has made the V-model redundant. Several models like Prototyping and Rapid Application Development (RAD) have proved superior to V-model[12]. Some of them have even been adopted successfully by companies. Job descriptions though have remained faithful to the V-model resulting in skewed job functions.
An offshore location with a different culture adds another dimension of misfit to this problem of ill-defined job descriptions. Jobs in Embedded Software attract the best Engineers at offshore locations. They are usually not satisfied with the quality of work leading to high rates of attrition. To remedy this problem, a complete overhaul of the development model as well as job descriptions/processes must be carried out.
6) Organization not prepared for offshoring:
To prepare companies for a global development environment, management usually adopts cultural sensitivity, diversity seminars etc. Although they serve as a vital first step towards preparing a company to operate globally several other aspects need to be considered before adopting a global development model. Two commonly overlooked aspects pertaining to software development are mentioned below –
- Ignoring or not accounting for negative attitudes towards offshoring
Nearly half the US population believes that the US is being harmed by the global economy; only a quarter believe it is benefiting. Nearly 70% of US residents believe that outsourcing hurts the US economy. Despite overwhelming evidence about the benefits of globalization, [14]majority of US residents and by implication employees are opposed to it. Yet few if any companies have conducted surveys to assess their employee’s willingness to participate in offshoring projects. With project management being performed by employees with negative attitudes, it is very likely that projects will fail.
- Not having proper project management at the point of offshoring
Similar to ignoring negative attitudes, very few project managers have been trained to handle offshore projects. Though offshore suppliers have tried to make things easier by providing project managers, there need to be trained personnel at the client company who can take charge of deliverables as and when required.
7) Failure to define a strategy for the offshore location:
When considering a project to offshore it is necessary to include it as a part of a Global Development Strategy (GDS). This strategy implies that some parts of the business or some processes will be handled entirely from an offshore location. Rather than select arbitrary projects to send offshore, a Global Perspective while framing any project will help a company become more competitive and eliminate some of the problems mentioned above.
The overall failure rate in offshoring Embedded Software projects has been enormously high. Yet the pressure to include new features in Cell Phones, Navigation Systems, Cars etc. in a cost effective manner are forcing companies to look at offshore locations. Avoiding the common mistakes mentioned above will likely make life easier for both offshore location as well as the client company.
[1] http://www.matimop.org.il/newrdinf/company/c142.htm
[2] http://www.koramangala.com/korasoft/2000/03.htm
[3] http://www.russoft.org/docs/?doc=54
[4] http://www.managementlogs.com/2006/03/reasons-for-offshoring-changing.html
[5] http://www.thehindubusinessline.com/ew/2005/05/09/stories/2005050900060100.htm
[6] http://www.galorath.com/tools_sem.html
[7] http://www.processimpact.com/process_assets/srs_template.doc
[8] http://blog.brodzinski.com/2007/08/dont-look-back.html
[9] Software Specification and Design: An Engineering Approach, By John C. Munson,2006
[10] http://ky.essortment.com/jobstraining_rshn.htm
[11] http://www.microcontroller.com/wp/DeviceDrivers/device_drivers.htm
[12] http://www.stylusinc.com/Common/Concerns/SoftwareDevtPhilosophy.php
[13] http://www.pollingreport.com/trade.htm
[14] http://www.cfr.org/publication/7749/trade.html
