The challenges inherent in making an ERP system available all day, every day are daunting, but Texas Instruments proves it can be done.
Texas Instruments Inc. has had quite a time of it the past few years. Company executives have worked hard to transform their firm from an organization that manufactures products to one that provides solutions. As part of its massive reengineering efforts, Texas Instruments also has made great strides toward becoming a truly global company.
A major part of the reorganization centers around making Texas Instruments' systems available, reliable, and global. To do that, the Dallas-based company has embarked on the long trek toward making its enterprise resource planning (ERP) system, databases from Oracle Corp., of Redwood Shores, Calif., and a host of middleware available 24 hours per day, 7 days per week, 365 days per year.
| IBM servers lead the pack |
Competitive Reliability, Availability, Serviceability, and Cluster Features and Functions A study of UNIX server reliability last year found that IBM's equipment slightly outperformed hardware from other leading vendors(Competitive Analysis of Reliability, Availability, Serviceability and Cluster Features and Functions, D.H. Brown Associates Inc.)
Source:D.H.L.Brown Associates Inc.
Today, Texas Instruments is in the final stages of rolling out its entire ERP environment on a 24x7x365 basis. The system, from SAP AG, of Walldorf, Germany, runs on clients around the world connected to Palo Alto-Calif., based Sun Microsystems Inc. E1000 servers running UNIX. It supports about 5,000 people worldwide across a WAN. It's one of the most ambitious SAP environments attempting to go to a true 24x7x365 model, according to SAP.
Facing the reality of 24x7x365
It's widely accepted that more and more companies plan to move their ERP systems to a 24x7x365 model. But the challenges inherent in making that happen--especially in companies with a global presence--can be quite daunting, as Texas Instruments found. Avoiding single points of failure, finding ways to upgrade and maintain the system without losing up time, and devising a proactive monitoring system are just a few of the challenges these companies face.
"If a company has distributed personnel and systems [and wants to do 24x7x365], that can be a huge hurdle to get over," says Kevin O'Malley, senior marketing manager for Oracle Applications. "The number of people you have around the world, the number of systems, the amount of staff required to support those systems, the number of third-party products your system uses, the number of databases you are running, and the number of different vendors you are dealing with, all play a role."
Phil Coup, vice president in charge of Texas Instruments' open systems transition, acknowledges that his company couldn't do it alone. In fact, the company's assembled team, which includes Andersen Consulting, of Chicago, SAP America Inc., of Newtown Square, Pa., Sun Microsystems, and others, is crucial to the success of the project.
"There are a lot of different variables, and using an entire team means that everything will be addressed," Coup says. "One challenge has been the Sun environment itself, because running an enterprise-level ERP system globally is new even to Sun. So having Sun on the team was important."
Coup says Andersen Consulting has been the biggest outside contributor, helping to determine the best way to run a UNIX environment at an acceptable level of reliability and, most importantly, devising a plan to manage all of the middleware that runs with SAP on the UNIX environment. Texas Instruments runs a plethora of middleware tools and decision support applications on an Oracle relational database, including Houston-based BMC Software Inc.'s PATROL for network monitoring; supply chain-planning system from i2 Technologies Inc., of Irving, Texas; a Java scheduling system from New Dimensions Consulting Inc., of Princeton, N.J.;and a file-management system from VERITAS Software Corp., of Mountain View, Calif.
"It's really complicated," Coup notes. "At the application layer, you've got SAP and different bolt-on functions such as Electronic Data Interchange (EDI). Then at the infrastructure level, you have the Sun Microsystems UNIX servers and all the middleware that is scheduling and running the jobs and monitoring the networks."
| NT clustering slowly gains credibility |
For those companies choosing to run their ERP systems in a Windows NT environment, clustering is likely to be a challenge--at least in the short term.
|For organizations worrying about keeping their ERP systems 24x7, they worry about whether NT is a basket strong enough to put all of their eggs into, says Sam Wee, a partner at Benchmarking Partners, a Cambridge, Mass., consultancy. And there is a real reluctance on the part of very large organizations to put all of their ERP eggs into the NT basket. |
Microsoft Corp., of Redmond, Wash., is well aware of the concern and is taking steps to remedy the situation. SQL Server 7.0 is more robust than previous versions, and Microsoft promises that the next version of Windows NT, dubbed Windows 2000 and due sometime within the next 12 months, will have the upgraded features of today's WolfPack (code name for an engineering project for clustering NT 4.0.) and Microsoft Cluster Server.
All of this is likely to help but won't necessarily solve the problem, says Neil Barton, head of the client/server computing practice at Compass America, a Reston, Va.-based management consultancy specializing in business improvement.
"It will help, but you still need to have a relational database management system that hooks into the cluster server," says Barton. "And on top of that, you have to have an ERP system that will work with that particular version of that particular RDBMS. And not every RDBMS works with WolfPack, while not every ERP system works with every RDBMS. It's quite complicated."
Even ERP and hardware vendors are leery. Hewlett-Packard Co., of Palo Alto, Calif., and Oracle Corp., of Redwood Shores, Calif., which last year announced an agreement to provide high-availability solutions through an integrated hardware-software bundle, limited their agreement to UNIX-based systems. At the time, an Oracle spokesperson declined to promise high availability on the NT platform, preferring instead to wait until Microsoft had further proven itself in that arena.
Others say all of the brouhaha over clustering is a moot point. "It's best to install and implement an ERP system with the capacity to meet the needs of the enterprise for the foreseeable future and to grow into it rather than adding to the system by clustering. Clustering is a Band-Aid solution," says Michael Charchaflian, president of Automation & Information Planners Inc., a Worcester, Mass., consulting firm specializing in productivity and process improvements. "While it may be necessary in the short term, clustering is less efficient than purchasing a system that will meet your company's long-term needs. Clustering should not take place by design."
Perhaps the biggest challenge has been in the change-management area. "We have talked to other companies running high-reliability UNIX environments, and they have told us that the change-management process is one of the most important things to a high-reliability environment, and we have found that to be the case," Coup says.
To face the reality of 24x7x365, Texas Instruments technologists meet each day to detail the various changes that are being requested or are needed from all of the various constituencies. Eventually, Coup expects the group to meet weekly or monthly, but for now change management necessitates a daily meeting.
Taking the plunge
Once you have decided to go 24x7x365, it makes sense to analyze each part of your system, optimizing it for maximum reliability and availability.
To truly succeed, you must focus on a plethora of inter-related tasks, including retraining administrators and help-desk personnel, negotiating special service and support agreements with your hardware partner, extending your system management guidelines so they map with the new requirements, adding an emergency computer center with a stand-by ERP system, and adapting your change management to the new requirements.
Experts recommend starting with your hardware. Most ERP vendors work closely with hardware vendors and integrate their software directly into the hardware platforms. At this point, most hardware vendors have 99.99% availability of their platforms, "and we work closely with them to make sure they have high up time," says Oracle Applications' O'Malley.
Redundant systems and spare backups also are an important part of the hardware equation. "If you want to run a 24x7 shop, you have to make sure that your service level ensures that you have spare backups on site or a commitment to get spare backups very quickly," SAP America's Alfano notes. "The challenge is to ensure that you are working with your selected vendor to make sure it has fault-tolerant and fault-resilient components that will allow you to provide uninterrupted service to that next layer." "A lot of these fault-tolerant infrastructural things are becoming increasingly common and increasingly relied upon," says Sam Wee, a partner at Benchmarking Partners, a Cambridge, Mass., consultancy. Having mirrored disks and a full backup system on site is part of the equation, but Wee says it's also crucial to have redundancy all the way out to the desktop and make sure there is no single point of failure. "You have to be extra sure that the basket will be resilient enough, because all of your eggs are now in a bigger basket."
Today, for the most part, ERP systems are being implemented in client/server environments. And like ERP systems, client/server architectures consist of multiple components--unlike the mainframe. But while that may have been an impediment as little as five years ago, the concerns have slowly vanished, Wee says. Today, UNIX-based client/server ERP systems use tools and utilities comparable to those that exist in the mainframe world, giving these environments high availability. And because the ERP system tends to ride on the operating system, issues regarding client/server environments are kept to a minimum.
For most companies, regardless of what is back at the datacenter, there has to be some type of upward movement to a more distributed set of tools as part of the ERP implementation. "At some point, a client/server ERP application requires that you have a process in place for not only maintaining the server, but for maintaining the clients as well," Wee says.
After you can be sure your hardware can stand the test, it's time to turn to your software. As Texas Instruments' Coup found out, making sure your ERP software works well with other software installed on your system is very important. That's where a consultant can really come in handy.
Oracle's solution for keeping its systems functioning and operating smoothly with other software is the Internet. Its browser-based interface, which is part of the new Oracle 11, has all of the functionality of its ERP system, but all of the code has been moved off the desktop.
"One of the areas where you see a lot of downtime is the desktop, because desktops are relatively unmanageable. End users have the ability to alter the environment by adding software or configuring something that may break another piece of software," says O'Malley from Oracle Applications. By moving to the Internet, vendors are attempting to remove the potential for damaging major programs within the application, some experts say.
Moving significant portions of the ERP system to the Internet also helps companies upgrade software. When a patch must be deployed or a new release installed, systems no longer must be shut down for a weekend. Instead, when a user connects to the system, he or she will be running the newest software within minutes.
Paying the price
|Reliability investment and return variables |
| Neil Barton, head of the client/server computing practice at Compass America, a Reston, Va.-based management consultancy specializing in business improvement, notes that the following variables can affect a company's 24x7x365 investment and return: |
| ||The existing infrastructure in place to support full-time operation. Large datacenters in particular are often supporting continual application service for some user sets and are able to leverage that investment to extend the service to others. |
| ||The mix of different workloads you decide are worthy of 24x7x365, and how sophisticated those applications are in supporting high-availability tools. (High availability software watches over one server, detects a failure, and automatically kicks in the standby.) |
| ||How effective your operation is today in consolidating your servers and fully utilizing your server capacity. |
| ||The number of discrete technologies you need for 24x7x365 operation. For example, if all your database servers are UNIX/Oracle, you can set this up once and largely copy the solution for subsequent servers. Whereas if you have to set something different up for Windows NT/SQL Server, and then something different again for AIX, it will take more time and cost more in software tools. |
| ||Whether any of your workloads can utilize the redundant server capacity for batch tasks, offline database backups or re-organizations. This reduces your long-term running costs by increasing the utilization of your servers. |
| ||The amount of storage your 24x7 applications are using. |
| ||The frequency of write-to-disk rates of the application's core relational database managers holding the ERP data. Demand for high update rates in an OLTP application may rule out RAID-5 for performance reasons, leading you toward RAID-1, which costs more. |
Once you have decided to go 24x7x365, it makes sense to analyze each part of your system, optimizing it for maximum reliability and availability.
Some say no amount of money is too much to ensure constant availability and reliability, while others caution that it makes sense to evaluate your needs and your environment before plunking down what can amount to tens or hundreds of thousands of dollars.
"The cost impact of increasing availability can go either way," says Neil Barton, head of the client/server computing practice at Compass America, a Reston, Va.-based management consultancy specializing in business improvement. "Folks operating in the 60 to 150 hours/week availability space score generally round about the .8 to 1.2 mark on a cost per utilized MIPS basis." According to Barton, some companies have been able to lower their long-term costs by consolidating their server workload, centralizing support, and deploying backup server cycles, rather than under-utilizing those servers.
What it all boils down to is the scope of the deployment, SAP America's Alfano adds. "It depends on the size of your business and how much ERP you are running. You have to evaluate the total cost of all components, all of the people, and all of the service levels required to take your ERP system from the standard non-24x7 to 24x7x365. Companies have to analyze exactly what it costs them to be down, either on a per-second or per-hour or per-day basis, with their ERP system."
Texas Instruments' Coup points out that around-the-clock up time requires an investment in each of the areas involved. "In the hardware area, we had to create parallel environments for failover capability and run dual networks to all of our sites so if one of the lines goes down, there is still a link into the site," he says. "Then there is the cost of the middleware software that runs the environment. And the skillset to run it [in 24x7x375 mode] is different."
In early 1999, Texas Instruments was running the SAP finance and procurement modules with 5,000 people worldwide on a 24x7x365 basis. Within a few months it plans to complete installation of i2 Technologies, global supply-chain planning system and SAP's sales and marketing modules.
These days, computer technicians at Texas Instruments still take the system down during the weekends, and scheduled weekend downtime is expected to occur for the next few months as employees get used to the new paradigm.
"But by the end of the year, we have an operating target that we won't have more than two hours per month of downtime," Coup claims. //
|Karen D. Schwartz is a freelance writer specializing in business and technology. Based in the Washington, D.C. area, she can be reached at email@example.com. |