Successful failover test ensures the stability of patent system applications at the USPTO
Blog by Andrei Iancu, Under Secretary of Commerce for Intellectual Property and Director of the USPTO, and Jamie Holcombe, Chief Information Officer of the USPTO
Illuminating the corridors of the United States Patent and Trademark Office (USPTO) data center in Alexandria, Virginia, are countless routers, switches, wires, and storage systems neatly organized on rows of server racks.
The blinking LED lights and soft whirring of cooling fans form an essential part of the USPTO’s digital backbone—one that supports ongoing patent and trademark examinations and daily business operations throughout the United States. Through this system, thousands of vital computer processes are executed every second of every day to enable the work done at America’s “Innovation Agency.”
Until recently, most of this capability lay within the walls of USPTO headquarters in Alexandria. However, we determined in 2018 that this potential single point of failure was no longer tenable.
In August of 2018 we experienced a multi-day outage of the Patent Application Locating and Monitoring (PALM) database. Afterwards, we committed to stabilize and modernize the USPTO’s IT infrastructure, which was long overdue. Significant portions of our infrastructure were antiquated and prone to failure under stress, and we lacked meaningful redundancies to mitigate the consequences of such failures.
We began the upgrading process by conducting a top-to-bottom review and mapping of all IT systems. This included engaging outside experts to assess the infrastructure, processes, and organization. We then prioritized the needed improvements, and we set to work. For example, last year we upgraded to a new server platform for the main patent processing system. This new platform is 1,000 times faster, 20 times more efficient, far more stable, and less prone to failure.
Critical to our stabilization effort is our team’s addition of redundancies – standby systems that enable “failover.” In a failover setup, redundant systems at offsite locations run simultaneously alongside the primary system. If the primary system fails, the standby system takes over, providing virtually uninterrupted support to system users. In addition to adding failover servers, we activated at the alternate site a second 10 gigabit-per-second (10G) Ethernet computer networking circuit, deployed additional active and stand-by databases for key processing systems, and automated our deployment process for the primary and backup locations.
To test the USPTO’s failover capabilities, on July 2nd and 3rd of this year, our IT team executed a planned outage of the servers housing the Official Correspondence and the Docket and Application Viewer. The files contained in this system are some of the largest, most important, and most accessed IT applications at the USPTO, used for reviewing, filing, and prosecuting patent applications.
The outage test went as planned, with no issues. The USPTO’s new standby system at the offsite location remained fully operational during the test, with a rapid and seamless transition of the applications from the primary data center. All data was updated in real time and remained secure.
Following this failover test, we then switched back to the primary data center, again with no issues.
Our IT team is now moving forward to fully automate the failover system, and we plan to execute similar failover tests several times throughout the year. All along, of course, we continue to upgrade various other hardware and data infrastructure.
USPTO has come a long way since the PALM outage, but we know there is still much more work to be done. We also understand that these upgrades will take time and require additional resources, and that we will encounter more hurdles along the way. We will stay the course.
Most importantly, we are committed to ensuring these essential IT systems are reliable for the examiners and users working to further the vital role that innovation, entrepreneurship, and intellectual property play in meeting the great challenges of our time.