British Airways (BA) is one of the world’s leading scheduled international premium airlines, annually carrying around 33 million passengers to over 150 destinations worldwide and transporting roughly 800,000 metric tonnes of cargo. It employs approximately 36,000 people and has a fleet of some 240 aircraft. As one of the world’s longest-established airlines, it has always been regarded as an industry leader.
BA’s executive management team recognizes that data is truly essential to the company’s competitiveness. Carrying more than 33 million passengers a year, the airline is careful to make certain that it captures commercial data such as customer reservation and passenger information, effectively.
The company uses the data across hundreds of vital business processes. Ensuring its quality can be a challenge, given the multiple sources of data and the different standards and formats, but it is essential to effective operations.
“Given the importance of good data to our business, we recognized the need for an enterprise-wide data governance strategy,” explained Paul Shade, head of business intelligence at British Airways.
A data governance manager was appointed, together with senior staff members from each of the key commercial functions such as marketing, bookings, customer service, sales and finance. Their role was to lead the effort for each area and champion the cause for data governance. Each business area also trained a staff member to take a "data defining" role; data stewards able to understand both the business need for data and the technical/IT aspects of managing it effectively. “Part of the data governance strategy was to review how we assured data quality across our business,” said Shade.
The data quality assurance review established that legacy data was stored in many different formats, held to different standards and was, despite existing data quality practices, of varying levels of cleanliness. Live data feeds too, such as bookings and passenger lists, did not always meet the high standard of data quality expected.
A significant observation made was that while data cleansing practices were in place, they were ‘point solutions’ implemented locally for the business area requiring the data, rather than holistically, at an enterprise strategic level. This meant that there was no single standard for data quality or method of achieving it across the business. Different rules were being applied in different business areas and a variety of tools and manual approaches were being used to cleanse the data. This was not especially efficient and the distributed nature of control inhibited central governance.
Additionally, executives had little means of judging the quality of the data supporting key business processes or feeding into their decisions. Also, management wanted to be alerted in the event of any long-term decline in data quality in key areas, especially if thresholds were threatened, but providing such trend information was proving to be difficult.
Recognising that it would benefit from a more strategic and centralised approach to directing data quality, BA chose to evaluate a number of data quality software solutions from leading vendors.
“One of the reasons for which we selected Trillium DQ was because it is a true enterprisewide solution from a pureplay vendor recognised as a market leader by Gartner,” said Shade. “Trillium DQ scales to hundreds of millions of records and is also able to work seamlessly across multiple systems and data formats.”
Trillium DQ is a unified software platform consisting of a set of products that, together, provide a complete solution for data quality discovery, understanding, improvement and monitoring. These are features fundamental to BA’s data governance strategy as well as to immediate requirements.
“Trillium DQ significantly outperformed the other tools when tested against our data in trials,” added Shade. “The software highlighted unexpected data issues, spurring a raft of demands for its speedy implementation to identify and resolve these and other potentially ‘unknown’ problems!”
BA decided to first focus its data quality project on three years of historical customer reservation data; approximately 100 million records equating to around 4.5 terabytes of data held in a Teradata data warehouse.
It defined three main implementation phases; discovery, improvement and monitoring.
Phase one of BA’s enterprise data quality implementation involved the use of the Trillium DQ data discovery and profiling tool. Trillium Discovery was applied to the airline’s bookings data to reveal formats, structures and inconsistencies, missing information and other quality errors.
Once this information was understood by the data teams, they were then able to collaborate with both the business and IT teams to agree on the necessary standards and rules for standardising, cleansing, de-duplicating and matching BA’s booking data.
Phase two first entailed the airline checking its own rules definitions with the very broad set of rules already provided ‘out-of-the-box’ in Trillium Quality, Trillium DQ’s automated data quality improvement tool.
“Trillium Quality comes with a very extensive set of built-in rules for data quality improvement, saving a huge amount of time,” said Shade. “The built-in rules have already been proven many times with other Trillium customers, so you know you can trust them.”
Of particular value to British Airways was that Trillium Quality enabled authorised users to fine-tune supplied rules as well as define custom data quality rules specific to their business needs.
With the rule sets confirmed, BA integrated Trillium Quality into its systems. Initially deployed in batch mode against legacy bookings data, BA targeted the improvement of its UK, US and Canadian customer address data.
After standardising, cleansing, validating (including against postal authority files), and then de-duplicating the data, BA was able to load it back into its data warehouse to serve commercial applications across its enterprise with reliable data.
“Trillium DQ is integral to the creation and accuracy of BA’s single customer view,” said Shade. “It’s essential to customer service, marketing, retention and loyalty and to business decision making.”
In phase three, Trillium DQ is enabling BA to monitor data quality metrics over time. Dashboards allow both business users and IT professionals to view metrics and rapidly assess, monitor and control the quality status of the airline’s multiple data areas. This includes bookings, ticketing, customer and market intelligence data.
“Using Trillium DQ, we are building a data-quality firewall,” said Shade. “This ensures only clean customer data resides in our data warehouse and reaches our business processes and systems around the world.”
“There is increased recognition of the importance of commercial data to the organisation and greater attention to its quality,” said Shade. “Our employees have more trust in the data they use for making decisions and running operations.”
In terms of a return on investment, Shade is very clear on where the returns are being seen.
“Our data analysts can deliver more accurate analyses more quickly and that supports faster and better strategic and operational decisions,” said Shade. “More relevant marketing communications are helping optimise response rates, and by understanding our relationship with each customer we can improve customer service.”
Shade continued: “We expect new applications to be implemented with reduced risks of delay or cost overrun since we can be more confident of smooth data integration and ETL processes.”
One mission critical application at BA is its airline Transaction Processing Facility which runs processes such as booking, seat reservations, ticketing and billing. It dates back to the 1980s and will need to be replaced by 2013. The assured quality of source data will be truly essential to the data migrations, integrations and ETL processes that will support successful implementation of the new system.
“Our data governance and data quality strategies are working well, and we’re planning on extending them right across BA,” concluded Shade. “Data is as much an asset of our business as our aircraft, our routes and our brand; the quality of that data is everything to us. Our investment with Trillium DQ is integral to our overall enterprise-wide data governance strategy.”