U.S. DEPARTMENT OF COMMERCE

PATENT & TRADEMARK OFFICE

- - - - - - - - - - - - - - - - - - x

In the matter of the:

PUBLIC ADVISORY COMMITTEE FOR TRADEMARK AFFAIRS MEETING

- - - - - - - - - - - - - - - - - - x

Monday, November 2, 1998

Patent and Trademark Office
South Tower Building
Arlington/Alexandria Room
2900 Crystal Drive
Arlington, Virginia

The Meeting of the PUBLIC ADVISORY COMMITTEE FOR TRADEMARK AFFAIRS convened, pursuant to notice, at 10:00 a.m. on Monday, November 2, 1998.

APPEARANCES:

Committee Members:

GARY KRUGMAN, Chairman

MARILYN DARE

MARK I. FELDMAN

SHELDON H. KLEIN

JIM JOHNSON

TED HARPER

KIMBLEY L. MULLER

PATRICKP. PHILLIPS

JOHN P. RYNKIEWICZ

ROBERT W. SACOFF

SUSAN UPTON-DOUGLASS

PTO Staff:

CHRIS SIDOTI

CHRIS KEIRS

JOHN JENNISON

LYNNE BERESFORD

ROBERT M. ANDERSON

J. DAVID SAMS

SHARON MARSH

NANCY OMELKO

Also in Attendance:

TODD DICKINSON, Deputy Commissioner

HOWARD FRIEDMAN, President, NTEU 245

BRIAN USILANER, Westat

C O N T E N T S

INTRODUCTORY REMARKS

STATUS REPORT ON ISSUES DISCUSSED AT COMMITTEE MEETING OF APRIL 20, 1998

LETTER OF PROTEST PROCEDURE

GRADE-LEVEL OF EXAMINING ATTORNEYS

REPORT ON CUSTOMER SURVEY

OFFICE OF TRADEMARK QUALITY REVIEW REPORT

TRADEMARK LAW TREATY

PUBLIC ACCESS TO PENDING APPLICATION FILES

TRADEMARK TRIAL AND APPEAL BOARD

FEDERAL REGISTER NOTICE ON PROPOSED INTERNET USAGE POLICY

DESCRIPTION OF GOODS AND SERVICES IN TRADEMARK APPLICATIONS

(10:08 a.m.)

INTRODUCTORY REMARKS

MR. KRUGMAN: Folks, it's a few minutes after ten and we'd like to get started. I'm Gary Krugman, chairing the Public Advisory Committee Meeting and I represent the D.C. Bar, and Bob Anderson has some remarks to make and he has to leave at 10:30. So we wanted to get started right now and let Bob do his portion of the morning session.

Without any further ado, I'm going to turn it over to Bob Anderson.

MR. ANDERSON: I'd like to welcome everyone to this meeting. Todd Dickinson, the Deputy Commissioner, was supposed to make opening remarks, but he hasn't shown up yet. I talked to his assistant just before I came down and he hadn't arrived at work yet.

At about 10:30, I have to leave to go down to NPR to talk about trademark quality down -- oh, here's Todd.

DEPUTY COMMISSIONER DICKINSON: Welcome, I guess. My name is Todd Dickinson for those of you who don't know me. I'm the Deputy Commissioner. I just want to take the opportunity to welcome the Trademark Advisory Board here. It's very important to our mission of the Trademark Office to have you all share with us a lot of the issues and concerns that you have.

I had the opportunity, I guess, to address you all about six months ago and I'm pleased to be back. We've obviously had some changes, which I know will probably be addressed today. We have obviously had a change at the top at the Trademark Office. Commissioner Hampton has resigned and is in the private sector now.

We will hopefully be -- we've actually already begun to do a search or consideration of individuals to fill that vacancy and that will be ongoing over the next couple of months. We are fortunate, though, to be ably led in the interim by Bob Anderson and his team, so I'm very pleased that they are here. We don't have a sense yet of how long that recruitment process might take and if my own confirmation is any indication, that took about a year, so we'll see what happens in that regard.

I think you'll also talk about a number of the things that concern you today with regard to the twin goals of pendency and quality. We certainly want to make sure that as we push forward to try to achieve the pendency goals which we've set for ourselves and for the administration, we don't want to lose sight of the equally important goal of maintaining a consistent level of quality.

We've recently conducted a review of that quality process as well and I think we're comfortable -- we're pleased actually with the results and hopefully, we'll be able to share some of that discussion today.

We've also, since our last meeting, taken several other initiatives, one of which I'm particularly proud of which is the placing of the Trademark -- the Patent's as well, but the Trademark's first data up on the Internet. All of our Trademark database is now up, back to, I guess it's '76 I think, right? Full database -- the patent folks only go back to '76. We've got the full Trademark database up.

Both images and text are up now. They're accessible through our web site, WWW.USPTO.Gov and we're very pleased that that has been achieved. We've gotten good feedback on that. As we discussed last time, there was some feedback from you all about whether we should charge or not charge for that service.

We took away from that previous meeting that you wanted us to charge for it. We went a different direction, and if you still believe that we should be charging a fee for that service as opposed to doing it for free, we probably should hear that; though, I will tell you most of the feedback we've gotten is positive in terms of offering the service for free.

The other opportunity, which I know Bob will probably go into in greater depth, is our electronic filing program. It's now possible to file your trademark application electronically and pay for it by credit card, something which Bob took the lead in and is largely responsible for happening -- not only happening, but happening on a timely basis and I'm very pleased that we've accomplished that.

So those are some of the things that we've done and we, again, greatly appreciate the opportunity always to get your feedback and if anybody has any questions or anything, I'd be happy to take them, but I don't want to disrupt your program, Bob.

(No audible response.)

DEPUTY COMMISSIONER DICKINSON: Okay, I also see Judge Sams out here. We should also acknowledge the fact the TTAB is doing very well. We will soon -- I guess we are now up to full complement of judges. We've added three additional judges during the last fiscal year, the last of which, Dave Bucher, just came on.

That, of course, unfortunately leads to a vacancy because he was the Deputy Assistant Commissioner for Policy, but we'll also look to fill that vacancy as well. I know that Dave and I had the chance to review the TTAB's progress this past year and it's been very strong and we're pleased by that.

MR. ANDERSON: Todd mentioned most of what I wanted to say. As you know, Phil Hampton left on October 15th. As Todd indicated, we are looking, or the Agency will be looking for replacement.

I do want to mention TLT was signed on October 30th. We do not have a public law number yet. The implementation time for TLT will be one year. There will be some discussion regarding TLT later today. We need to get a rules package going fairly quickly and we would appreciate comments regarding proposals we have on the table.

Electronic filing began on October 1st. Based on the pilot which started last November and ended on October 1st, we had only had about 250 filings and we didn't have a lot of optimism that it would be all that well received. However, since October 1st, I was told this morning we've had over 830 filings with the electronic application.

One interesting thing that happened that initially, the bulk of the applications were coming from individuals. Last week, more applications came from corporate America than came from individuals, so there appears to be an increasing interest in corporate America in using the electronic application.

As Todd mentioned, we do accept credit card payment on it and thus far, most of the applications received since October 1st have used credit card payment as opposed to deposit account. Unfortunately, even though we process those applications fairly quickly and give feedback quickly, they will still sit in line for a fairly long period of time until they get a first examination.

For right now, that's about all I have time for. Both Todd and I need to get downtown to the old executive office building for a meeting. We will be leaving in just a few minutes. I think the next topic on the agenda --

DEPUTY COMMISSIONER DICKINSON: One more thing. One issue that I'll maybe just touch on briefly because you will talk about it in some depth is the new space consolidation project which has been ongoing for a number of years now.

As you know, we are spread out very thinly here at Crystal City. This is the farthest reach in one direction. If you go down to where the Board of Patent Appeals and Interferences is, it's about a mile and two-tenths the other direction. We have shuttle buses that take everybody around.

What we would like to do is consolidate down into no more than eight buildings. As you probably know, we have had this RFP out for some time to allow us to do that. We've narrowed down our search to three developers who will develop the space and then lease it to us.

The government does not allow us to build a new facility and own it because all that cost would be on our bottom line this budget year and when you're trying to balance a budget, putting an extra half a billion dollars on the bottom line is too big a bite to swallow. So we will continue to lease.

The project was a source of some controversy through the summer and early into the fall. Some watchdog groups have charged that we are reckless, I suppose you'd say, in their belief in terms of how spending might be made on furnishings and other things. Let me say if that is -- if you read a recent Washington Post article or Washington Times article, you'll know that that is largely a political debate as opposed to a real debate. We are not spending the kind of dollars on the kind of things they talked about in that article.

The project was also the subject at a couple of points of possible continuing study at the request of Congress. All of those amendments were beaten back. The last one we thought might come along was in the Appropriations Bill. That was not included. As a matter of fact, the Appropriations Bill went in a different direction. It lifted -- or it did not included the so-called "Brown-back Amendment" which would have capped the amount of money we could spend on furniture and build-out and other things, saying that it was premature.

So we are in relatively good shape relative to that project going forward. The final decision will be made on which of the three bidders that remain, probably in December. The three sites, as you may know, are here in Crystal City -- our current landlord is one of the finalists -- down in the Carlysle area, which is a new development in I guess it's the King Street metro stop and then the Eisenhower metro stop, which is just beyond that, both of which are south of here, but not too far south in Alexandria.

There was a certain political competition between Alexandria and Arlington to try to get this project and I think that will have an impact. There are some concerns about neighborhood groups as well, which I think will have potential impact if one of the sites, in particular, is chosen.

We won't make the choice decision, just so you're aware. It will be made by a five-person panel of procurement specialists. We have one representative on that panel, that's Kaz Kazinsky, but the others come from either GSA who is largely responsible for the reality of the project, or there is also one representative from the Department of Commerce.

MR. KRUGMAN: Thanks, Todd, and thanks Bob. You better come back. The agenda that you got in the mail about a week or so ago has been shuffled around. The revised agenda, I think, is on the desk. The shuffling was only because of Bob's schedule in terms of having to leave and come back later.

The next item on the revised agenda is a status report on the issues discussed at the last Public Advisory Committee meeting. Sharon, were you going to do that?

STATUS REPORT ON ISSUES DISCUSSED

AT COMMITTEE MEETING OF APRIL 20, 1998

MS. MARSH: We're not going to spend much time on that this morning. In your package of handouts, one of our staff attorneys went through the minutes of the last meeting and identified all the issues that someone on the committee had raised and in the handout, each of those issues is listed and there's a brief summary of what the office has done.

Given our limited time, I don't think we should spend any more time on it this morning. Take a look at it. If you've got further questions, we can answer them later in the day.

MR. KRUGMAN: Okay, then let's go to the next item, unless anybody has a question now, which we'd be happy to entertain, but I know we just got these things a few minutes ago.

(No audible response.)

MR. KRUGMAN: The letter of protest procedure, proposed change in practice, one of the Public Advisory Committee members asked me to contact the office and have this put on the agenda, and Sharon, did you want to address this?

LETTER OF PROTEST PROCEDURE

MS. MARSH: Yes. Through Gary, we were asked to put letter of protest procedure on the agenda. Jesse Marshal is the attorney in our office who handles letters of protest and due to a family emergency, she's not going to be able to be here today, but I think we should go ahead and discuss the issue.

I don't know who wanted the issue on the agenda or what their concern was, but when it came up in the office, after some discussion, we think there's a good argument for eliminating this procedure. There is a short discussion of it that you can take a look at.

This procedure predates my time at the office. We started it apparently about 20 years ago and it was at a time when examining attorneys had very few resources, and it was very difficult for examiners to prove that a mark was merely descriptive or to show that certain channels of trade were related. So it was a way that third parties could provide evidence to the office that would help us sustain a refusal.

Over the years, the practice has continued. A few years ago, we tightened up the procedure a little bit and made the standard clear error. The letter of protest will not be granted unless it would be a clear error for the examiner not to make the refusal. You're also limited in submitting a letter of protest. We'll only consider it if the mark is in examination or within 30 days after publication.

Anyway, if you look at the statistics in the third paragraph, Jesse went through and looked at how many letters of protest she gets and what the issues are. In fiscal years '96 and '97, she got 1,200 letters of protest and only 175 of those were actually granted, and she goes through and lays out the issues and how many were filed for each issue.

It's a significant amount of staff time and money spent on this at a time when examining attorneys have access to much better resources through Lexus, Nexus, and the Internet. They generally don't have a problem finding the evidence they need to support a refusal, and there's some concern on the part of the office that really this is merely a way for third parties to try to interject themselves into the ex-party examination procedure.

So we thought we'd put the issue in front of the committee. Our proposal is that we eliminate the procedure all together, but we, of course, want to hear what your thoughts are.

MR. KRUGMAN: John?

MR. RYNKIEWICZ: Well, I'll admit I'm the one that asked to address this issue, John Rynkiewicz. I think the office needs to take a look, Bob, and we ask you to take a look on the behalf of Trademark owners and practitioners at this procedure.

I think the fact that it's taken a lot of time and there's backlogs is not a reason to get rid of it. It serves a valid purpose and I think the focus predominantly centers around likely the confusion issues, but we must take a look at the types of letters of protest that are filed.

Common usage in the trade, a few years back, the office asked people outside to submit evidence and materials that could help the office. I understand the Internet is available now to help in that respect, but there are types of issues and a few of our members, for example, have shown me some of the problems they've experienced.

I won't get into specifics but there is a detailed letter of protest that was filed dealing with color and it showed common usage in the trade by about six different competitors. The letter of protest was lost and then once finally examined after publication, the simple form letter said that there's no clear error by the examiner, and I think the detailed evidence showing this color, common color being used, deserved more than just a simple form letter that's used most likely in the -- likelihood of confusion situation, so I think there's an example.

Another one that came to mind was litigation. Serious federal court litigation was brought to the attention of the office and it was brought before the application was examined. The examiner allowed the application and there was still a delay in pulling the applications. I find there's another major problem. These cases are not pulled quickly.

In David's situation, there's letters of protests after publication. They're not pulled and then the extensions of time have to be filed and David's group at times is criss-crossing trying to find the file or the Office of the Assistant Commissioner has not pulled the file.

So I think the procedure of pulling the file, identifying and acting on these, you know, going to the heart of the issue, I think the likelihood of confusion is an interparties issue most of the time, but there are other important issues that need to be addressed, and that would be, for example, the common usage in the trade of color, litigation, and the office needs a way of getting that information presented to it.

To say that it's a clear error standard, I don't think an examiner would have necessarily this type of evidence of usage in the trade. I don't think an examiner would have evidence of a litigation going on in federal courts. So I think it's an important thing. It could be refined, but I think the office needs to really improve some of the processing with it.

MS. MARSH: John, on the litigation, what we were thinking there was that yes, there would be no way to stop it necessarily because we might not know about the litigation, but as soon as the mark publishes, an opposition can be filed, and a request to suspend the proceedings made to the TTAB, and doesn't that get you the same result?

MR. RYNKIEWICZ: Well, it's putting a lot of pressure, I think, on the -- as it now stands, the PMBP does say litigation is a valid reason for a letter of protest, so you take that away and say you must file in the opposition is creating an extra step, which I don't know if it's really necessary.

MR. KRUGMAN: Susan?

MS. UPTON-DOUGLASS: I wanted to also say that some of the letters of protest involve what we would consider to be dead hits that are somehow missed by the Trademark examining attorneys and I have, at times, looked at the search strategy and the file wrappers and you could see from the search strategy that there's no way they could have found this with the way that they were searching it.

It's like one letter difference that's a vowel and you know, maybe there were only 175 granted or whatever, but that's 175 oppositions that don't need to be instituted, and the files made up, and the wheels put in motion, and a summary judgment motion filed, or whatever, and to the extent it catches things that should have been caught, it's a very good idea.

MR. KRUGMAN: Bob?

MR. FELDMAN: I also think that these are not granted all that often, but it seems like there's a place for it and looking at those search strategies, we've seen instances where someone's filed for a two-word mark, our client owns something for one of those words, but the search was done only for the two-word marks.

It just seems to me that's a valid basis for a letter of protest. I also, for the life of me, don't understand why some of these have been refused. We've gotten some refusals of letters of protest in that exact situation. We have a famous one-word mark and somebody filed for two words. We filed a letter of protest and we get back denied.

But I do think there's a place for it even though they're not often granted. Yes, we could oppose every time if the letter of protest was not available, but it does seem like a useful procedure.

MR. KRUGMAN: Shelly?

MR. KLEIN: Just to add my voice to those who are opposed to doing away with the letter of protest procedure, particularly, the mention of the expense of approximately $25,000 a year. I mean, that could equal one opposition that has been saved.

So I think it is an extremely cost-effective means of short-circuiting a potential interparte proceeding, and I don't think that there was a comment here that it really could be viewed as being counter to the statute and regs because it's, in effect, an interparties of proceeding. I don't think it is because of the way you handle it, you only give the evidence and the factual materials to the examiner and then it's still the examiner -- it's still up to the office to refuse or go ahead with an application.

So I would urge you -- I think the idea of formalizing it is a great idea. There's nothing wrong with that.

MS. MARSH: Yes, that was our -- in the event that the group, as it sounds like you do, believes that this is a useful procedure, we think it ought to be codified. It's not in the statute, it's not in the CFR rules. It's just an internal procedure that's been developed over the years and so, we were proposing coming up with a proposed set of rules on the issue.

MR. KRUGMAN: Bob?

MR. SACOFF: First of all, let me apologize for being late. My flight just came in.

MR. KRUGMAN: You told me that it would be.

MR. SACOFF: Forgive me if I'm duplicating comments somebody else made before I got here a couple of minutes ago. Just quickly on the letter of protest procedure, I did see the proposed change of practice and the outline. I didn't realize it was a proposed elimination of it.

I think it's a useful procedure. It's really not used all that often in my experience, but mistakes happen from time to time and there ought to be this vehicle for bringing in an overlooked or an oversight to the attention of the examiner and -- I don't think it's substitute for interparties' proceedings, but frankly, I'm presently astonished to see that the cost of this procedure is being calculated as a little as 25,000.

To further what Sheldon said, I think 25,000 would be a very inexpensive contested opposition. So I think it should be retained.

MR. SIDOTI: Yes, Chris Sidoti, ABA. I would concur with everybody what they've said here. First of all, a letter of protest is a means by way a clear error could be caught very early on and the damaged party or parties do not have to wait for an application to go through a long process, particularly, if it's an ITU application.

Beyond that, we don't know what the impact will be on the TTAB if all of these letters of protest turn into oppositions and we might be creating more of a difficult situation in terms of backlogs at the TTAB.

I think rather than just getting rid of the system because of inherent problems, it should be improved and it should be, as you say, possible codification, formalization, and then maybe staffing up the procedure enough so that it could be more efficient than it has been.

But it's clearly the only way that the outside bar can weigh in on something that's very critical and with the change in technology and what have you and the massive hirings that are going to go on, it's going to possibly be even a greater threat of clear error down the road in the near future.

MR. KRUGMAN: Yes, Patrick?

MR. PHILLIPS: Pat Phillips, the Ohio State Bar Association. After I got the proposed agenda for the meeting, I circulated it around to several people. One came back on the letter of protest and they would love it to be somehow codified because they were in a total quandary of how to do it, which is the right way, what's the wrong way, and how can I have the best likelihood of getting it through?

Another individual came back and their concern was similar to what John had talked about initially. They felt they had a slam dunk on their letter of protest and got back the form letter saying thank you, but it's denied, and then they had to go back and tell their client if this is the policy of the Trademark Office or the way they're viewing this.

We would tell you your likelihood of prevailing in an opposition would maybe be slim and next to none. They would have liked to have seen a little bit more thorough review in that particular case.

MR. KRUGMAN: I mean, I know I don't use the letter of protest in my practice because as a practical matter, it's so cumbersome and it's so unlikely to be granted, but that doesn't seem to me to be a reason to not have it. It seems to be a reason to improve it and maybe formalize it and just make it better.

Does anybody on the committee have a contrary view? Does anybody go along with the office's proposal to abolish it just so we can be heard on that, or is it pretty much unanimous?

(No audible response.)

MR. KRUGMAN: It sounds like it is. Sharon, one thing I --

MR. JENNISON: Gary, if they could just speed up the letter of protest too. It's an awful long backlog on it. I think it's a three or four-month backlog reviewing them, so that would help as well. Get the results back to the examiner.

MR. KRUGMAN: I think everybody would be happy with that. One thing I just wanted to say on this meeting, the charter of the advisory committee is that observers also are entitled to comment about the agenda items and I don't want to forget that. So if anybody is sitting in the observation area that has a comment about the letter of protest procedure or the proposal to do away with it, I'd be happy to entertain some brief remarks if anybody has anything to say and that's true with this topic and any other topics later on.

(No audible response.)

MR. KRUGMAN: Well, Sharon, it sounds like we've spoken as one.

MS. MARSH: It looks like you've got a unanimous point of view. I will pass this information on to Jesse and the office will discuss it, but it sounds like we should prepare a set of CFR rules to cover this practice.

MR. KRUGMAN: It sounds like one of the main problems with it -- I know it is from my perspective -- is the time. We can keep it and we can formalize it, and we can codify it, but if it's still going to be a real cumbersome, time-consuming process, is there any way that this could be speeded up?

MS. MARSH: I don't know. This is a question that Mr. Anderson should get because it probably means adding staff on this issue and I just don't know if we can do that or not.

MR. KRUGMAN: Okay, well maybe we'll surprise him with that one later. Anybody else have anything? Sharon, did I cut you off?

MS. MARSH: No. John, did you have something?

MR. RYNKIEWICZ: John Rynkiewicz. Sharon, I think I've just looked at this over the last couple of years and I think it's gotten away from how it was examined. I think there's, in talking to different people in the office, we've been told that instead of just waiting until the examiner reviews, the letters of protest are kind of watched.

So if you have a likelihood of confusion, they wait to see if the examiner does, in fact, cite. I think that's getting away from what the letter -- I think you deserve an answer. A lot of us have watch services for important marks. You find these marks within weeks after they're in the database, and I think you're entitled to getting an answer, not five or six months later when it gets examined.

So I think the procedures of when and looking for clear air, depending on the issue, I think that they should just be looked at fresh and not seeing whether the office is -- because I think there's a desire that maybe they'll take care of themselves, if they're cited and it just goes away. I think it should be -- you know, you're owed an answer or a reply on the letter of protest and I think that should be looked at early on.

MS. MARSH: Okay, I will pass that information along.

MR. RYNKIEWICZ: Okay.

MS. MARSH: Gary, the next item on the agenda was also one of Jesse Marshall's issues related to identification of goods and services, but I think in the interest of keeping us on schedule, if we could delay that one until this afternoon, we have the consultants coming to talk about the customer survey results at eleven and I want to leave time for Howard Friedman's issue first.

MR. KRUGMAN: Okay.

MS. MARSH: So, if we could go to that issue now.

MR. KRUGMAN: All right, so we're looking now at number five on the revised agenda which is a NTEU proposal for instituting a non-competitive GS-14 for examining attorneys, and Howard, who is the NTEU representative, I guess, will speak to that.

GRADE-LEVEL OF EXAMINING ATTORNEYS

MR. FRIEDMAN: Yes, good morning. My name is Howard Friedman and I'm president of NTEU-245. We represent the Trademark attorneys at the Trademark Office. First off, I appreciate the opportunity to speak before the committee today.

I wanted to discuss primarily why it is in this committee's best interest, our customers' best interest, to support increasing the number of GS-14 examining attorneys, but before I discuss those particular benefits, I wanted to first review how a Trademark attorney handles an application, discuss the time constraints that we face when we examine a trademark application, review our present classification system, and discuss how we are rated.

This, as you all have seen, is a trademark application. It, by the way, does have a search in here that would pass the letter of protest test, or at least I think it was since I did the search.

I think it's important, as I said, before getting into the benefits to walk through, particularly for those committee members who didn't work at the Patent and Trademark Office and I know there's at least three on this committee who did, but primarily for those that didn't.

We get this trademark application, we do a search. As others have alluded to, obviously, we have a wealth of resources that weren't available in the past, which means if you're looking for a translation, you're doing a translation search, you're doing lexus/nexus evidence, you're doing surname searching conceivably over the Internet which is what some of the people in our bargaining unit do now, and in effect, you're doing a lot of their resource material even before you get to the final refusal because you want to let the applicant know as best as possible where they stand after they get a first office action.

In doing all those actions, you're under a fairly strict time constraint and I think it's important for the committee to understand what those time constraints are, and in that regard, let me focus primarily on the GS-13 and GS-14 levels.

If you're a GS-13 attorney and you're operating at the fully successful production rate, you need to turn around an examination of an application every 49 minutes. If you're trying to be at the outstanding rate, which is at 110 percent, your turn around time is 45 minutes, and again, that's to get the application, to review it, to do your search, to gather the evidence, and of course, produce the office action.

As I'm sure many of you know, particularly those who are practiced or were at the office, conceivably, you're spending a lot more time than that when you're going fine because you need to gather the evidence because you never know which of the people here in your brethren might be going an appeal before the TTAB.

I think if you think about that production rate and then you tie what 13s and 14s do, if you're a 14, the turnaround time is 46 minutes in order to be fully successful. The turnaround time is 41 minutes if you're a GS-14, a little more than two-thirds of an hour.

When you think about that production rate and you rate how attorneys are classified, there is what's known as the GS-905 series which in the Federal Government classification series is the correct classification for attorneys, and when you're evaluated or when they classify what the appropriate series are, they look at two factors: number one, the independence of your work, and number two, the degree of complexity of your work.

Depending upon whether you're a GS-9, a GS-11, a GS-12, or a 13, or 14, those impact upon how difficult the cases are that you handle and how independent of your work is. Is your work reviewed?

It's important to note that in looking at those particular standards, which are really the only standards that are supposed to be evaluated. Production is not a standard in the 905 series. Now, it has become a de facto standard in our performance appraisal plan, but as it stands now, when you look at the difference between a GS-13 at the office and a GS-14, it shouldn't be production, it's really only two issues: complexity of the work and the degree of independence.

If we think of those factors and then we look at our current performance appraisal plan, and I provided the committee with our current performance appraisal plan a couple of weeks ago, there is really only two differences currently between a GS-13 at the office and a GS-14.

One is the production level that I've discussed and two, in order to become a GS-14, a GS-13 has to go through a competitive selection process. Those are the only distinctions between a 13 and a 14 in our office. Thirteens handle the same complexity of work that 14s handle; 13s and 14s both handle intent to use cases, trademark applications, service mark applications, 1A, 1B, 44D, 44E.

If you're a 13 or 14 in law office 101 and I'm a 14 in 101, the 13s in our office handle the exact same trademark applications that I and other 14s handle -- no distinction, and there's no distinction in our path between a GS-13 and a GS-14 again, with the exception of production.

MR. KRUGMAN: Aren't the 14s supposed to have some training responsibilities of younger examiners? Is that supposed to be the difference?

MR. FRIEDMAN: A couple of years ago, that was one of the primary differences. Now, Gary, there are a number of 13s in our bargaining unit who train. There are also 14s who do training, but there are also 14s who don't do training. So, really, the lines of demarcation when it comes to 14s who train and 13s who don't have blurred and frankly disappeared. It's really up to the law office manager to decide which people in their office amongst the 13s and 14s are allowed to do training.

Basically, the office has taken the position that they're going to pick those people who they think are the most qualified to do the training.

Now, as a result, primarily because of the fact that there is really no distinction between a GS-13 and a GS-14, other than the two distinctions that I've mentioned, a recently concluded report by NAPA, the National Academy of Public Administration, which was commissioned by Commissioner Lehman, noted that there was a basis for a change in classification to the full performance level from a GS-13 to a GS-14.

Again, the report focused on two aspects and that was the complexity of the work that 13s handle is the same as the complexity of the work that the 14s handle, and similarly, or related to that, there is no review. A 13, just like a 14, does not have their cases reviewed for pub. I think maybe when some of the gentlemen here were at the office, even when they were a 13, they'd do a case, they'd put it on the shelf, and their managing attorney would review it.

That's not the case anymore. Thirteens and 14s work, you do your pub or your registration. It goes on the shelf, not to be reviewed by a manager and that's the end of it for that purpose.

I should note that when the NAPA report was made, the Commissioner was concerned about the internal equities in pay amongst ten major occupations. The report did an exhaustive analysis of these ten major occupations and this was throughout the PTO, including patent examiners.

There was one, and only one, position where the report noted that there conceivably was a change in basis for classification upwards, and of those ten classifications, the one that they said there was a basis is the one I've just noted; the trademark attorneys.

Otherwise, the report basically noted that all the other classifications were in accord with what the -- were commensurate with the responsibilities and duties of the people at the office.

So, I've tried to briefly -- before addressing where you fit in, I've tried and wanted to briefly go through what we do and to some degree how we're rated and I've touched not a lot on the PAP, but we are rated based our refusals, docket management, organizational effectiveness and practice and procedure as well as production.

I've gone through what the NAPA report has mentioned. I've tried to make -- I've been very brief, but gone through some of the things that we have to look at when we look at a trademark application and I've tried to tie all of that in to the production standard that we're under.

Where do you fit in? Well, clearly, both on things I've seen in writing, the testimony that INTA and other organizations have submitted on Capitol Hill, and based on the past meetings that I've been attending here in the last year or two, you have a very simple mandate.

You want cases handled as quickly as possible without compromising quality; fairly simple. Indeed, the office, as I'm sure Bob will be talking about later today, is under a Congressional mandate to have cases acted upon within certain time frames which unfortunately, are not close to being met.

One way, and it's not the only way, but one way to reduce pendency is to have the office's most experienced attorneys handle more of your files which would, of course, mean more GS-13s if not all GS-13s becoming GS-14s. Other benefits that would accrue to the people at this table and your clients would be reduced attrition, and I'm sure when Bob again talks about the statistics over the last six months of this fiscal year, people are leaving.

Three people have either announced that they're leaving, one of whom has left in the last couple of weeks, and the concern obviously is that the salaries -- one of the concerns is the salaries at the office are not keeping track of what, of course, have been the increases in the salaries in the Washington, D.C., New York, and other areas.

In addition to reducing attrition, actually, it has another benefit because if people on the outside know that people on the inside are being paid more money at the GS-14 level, conceivably that will attract more experienced practitioners on the outside to come to work to the Patent and Trademark Office.

Conceivably, the more experienced attorneys you have, the less of a need there is to hire a number of the new attorneys that will be coming on board that Chris had made reference to, and conceivably, as the Commissioner had noted, we'll be moving into new space over the next couple of years.

Perhaps the more experienced attorneys that stay, the less need because they'll be producing more, the less need you'll have to pay for office space. Remember, a GS-14 attorney is required to produce at the 1.3 rate which is compared to a GS-11 attorney which is at the 1.0 rate.

A GS-13 attorney is required to produce 30 percent more applications than a GS-11 attorney and is required to produce 8 percent more trademark applications than a GS-13 attorney.

I think other benefits would be by hiring a large number of attorneys, or in hiring a large number of attorneys, you have to be concerned about training. You have to be concerned about consistency of office actions. Obviously, the more experienced attorneys that say -- and I think this goes directly to your concern, Gary -- the more experienced attorneys you have to train the new people that are going to be coming on board in the next year or two.

Obviously, there is an additional benefit that the more experienced attorneys you have, the better quality and consistency of examination you'll have.

To anticipate some questions that the committee may have, how can we be sure that quality will not be compromised by having more files going out the door quickly:? First off, you'll have a better chance of having more experienced attorneys evaluate the files that you submit to the office. Our performance appraisal plan also calls for quality review. Three of our files are looked at. Random files are looked at every month.

Additionally, as we all know, published marks in the O.G. are reviewed. Trademark quality control, Chuck Conger's outfit, monitors examination quality, and as I think will be discussed after I move off, the recent survey results will also show frankly that the people here, and at least the people who have submitted the survey, are happy with examination quality.

What about cost? First off, the office does have another announcement out to hire more 14s. There were 19 that were hired in June. The office has a rolling admission announcement out and it's unclear how many they will be hiring between now and when the next announcement closes in December, but from a cost perspective, what you're talking about is the incremental cost of those GS-14s who would become GS-14s through other than a competitive process.

Also, as the survey results show, your main goal, I think, is to get the unexamined applications out the door, and according to the recent survey, which I believe will be touched upon, you were at least satisfied with the process time standards.

One of the most, or the most improved standard, compared to a 1996 survey, the most recent survey done before the '98 survey was the cost of trademark applications where your satisfaction level went up from 45 to 56 percent and I'm sure that's because there hasn't been an increase since December of 1993.

Finally, I think you'll get a return on your fees through higher retention, better quality, better training, increase of morale, and a potential reduction in new hires and the need for additional space; all factors which you probably can't put a tangible dollar to, but I think do have a dollar value to you and to your customer.

With that, again, I thank you for the opportunity to speak and I'll just leave open the floor to any questions people have.

MS. UPTON-DOUGLASS: I just had a question about how you envision this working, doing away with the competitive selection process. Would it then just be after you're a 13 for a year, or after you've handled a certain number of applications? Or, what would be the triggering event to be considered for this GS-14?

MR. FRIEDMAN: It's a good question. Clearly, as you probably know, before you become a 12 and before you become a 13, there's a poll of cases and if you have a poll of 10, you can't receive more than one statutory refusal that you missed, or three or more practice and procedure errors.

We've always taken the position, and we haven't gotten this far with the office that it's probably only fair that there be some type of similar, if not more stringent criteria so that maybe the way it would work is after you've been in grade for a year at a 13 level, there would be a poll. Maybe it's similar to the poll that takes place as a 13. Maybe the requirements are a little more stringent. We're obviously open to discussing them.

So, you and the office, and Bob, and Todd, and others feel comfortable that before somebody's becoming a 14, they've passed some type of criteria. So, it wouldn't be, you know, from our perspective, as much as we'd love it, we wouldn't envision a year going by and then you running to your manager and saying could you please process my 52 so that I can have my additional paycheck -- additional money in my paycheck.

MR. KRUGMAN: Mark?

MR. FELDMAN: I know that you've been pushing for this for a while and I'm just curious if someone could speak for the office as to why they've not implemented this until now.

MR. FRIEDMAN: Oh, I'd be happy to speak for the office, but --

(Laughter.)

MS. MARSH: I think we need to wait until Mr. Anderson is here. He'll be back after lunch.

MR. KRUGMAN: Okay. John?

MR. RYNKIEWICZ: Howard makes the case based on his facts, but I think it should be pointed out too just from practical standpoint, the patent side of the office for years has had a working grade 15, I believe, and inroads have been made to remove and I don't necessarily like this phrase, but the stepchild of the Agency, I think a lot of improvement over the years and I think this is one issue that's been looming for a long time.

You can argue well, you have to have a technical background, but on the Trademark side, you have to have a legal background. So, I don't know what one trumps the other, if at all, but I think for parity sake, it doesn't make sense that there should be a discrepancy of two working grades.

So, for that matter alone, without looking into the facts that Howard was presenting, I just think that that's an inequity that exists on the Trademark side within the common agency.

MR. SIDOTI: Yes, and if you could speak, I used to work at the Trademark Office in a galaxy a long time ago and I can tell you that there is a large group of examining attorneys who are senior examining attorneys who have not wanted to place themselves in the competition of the typical GS-14 appointments who are experienced examining attorneys who regularly mentor people and who do an incredible amount of work.

It seems to me that these kinds of people who are really in the trenches producing and helping the pendency who are mentoring people maybe informally or formally, should be rewarded so they stay with the office.

This is not just, it seems to me, a case of increasing grades for the sake of it, it's bringing the Trademark Office and the experts at the Trademark Office in parity with the outside. It's trying to keep people, and I agree, it shouldn't be totally automatic. It should be very stringent. It should be just the best cream of the crop, but it should be something that should be seriously considered.

I know examining attorneys now who I speak to who have been there for many, many years who have chosen just not to be subjected to the political scrutiny that often goes on with competitive appointments. But these people are doing an incredible amount of work and I think that that kind of work should be rewarded so we keep them and so that we create a work atmosphere at the Trademark Office where we have excellent people who will stay.

So, having worked here, having been on the outside, I think it's a no-brainer. I think it's something that we should support wholeheartedly.

MR. KRUGMAN: Jim?

MR. JOHNSON: This is Jim Johnson. I concur wholeheartedly with Chris Sidoti's comments. I think it was difficult working in the office knowing that on the patent side that the pay equity was so -- the disparity was so great and I think that we should have a process in which the best of the best do get promoted to 14. I just don't see any reason assuming that we can afford to do so that we don't do so.

MR. KRUGMAN: Howard, just out of curiosity, if the training requirements for the 14s have sort of blurred, vis-à-vis some of the 13s. Some of the 13s do training and some of the 14s don't. The competitive openings for 14s now, what's the stated reason for them being eligible to get a 14, other than just getting more money? Is there at least stated reasons that there would be training responsibilities?

MR. FRIEDMAN: Well, I have the criteria so probably the best way to respond is to read what the criteria is from the most recent vacancy announcement, or at least put it in capsule form.

One is -- there are three criteria. One is expert knowledge of trademark law, procedures and work processes, as well as market place issues related to the prosecution of trademark applications.

Two is the ability to examine a large number of applications at an exemplary rate of production and consistent high quality.

Three is the ability to communicate effectively with internal and external customers to provide accurate information about the application process, and to investigate problems and take appropriate action.

There is no mention of training in that particular vacancy announcement. There was in the announcement that issued in February, the prior announcement to this one.

MR. KRUGMAN: John?

MR. JENNISON: Howard, of the GS-13s you have now, do you still have the bonuses that are available if they go above their performance in amount of applications they examine?

MR. FRIEDMAN: Right.

MR. JENNISON: Is that still in place?

MR. FRIEDMAN: Yes.

MR. JENNISON: So, if you went to the 14 by right or by noncompetitive means, the office wouldn't have to mess with the bonus situation for all the 13s. There would be an ongoing 14 level throughout the year instead of having the large increase of production that's done at the end of the fiscal year or perhaps in March, is that correct? It would be more of an even production for the office throughout the year?

MR. FRIEDMAN: Well, I'd like to say yes, but I don't know if that would occur that way. I mean, I think if there are incentives to -- you know, if the office and therefore, its customers are looking for more even production instead of maybe the dips and flows that occur during the year, you know, maybe a way to tackle that is to have a similar bonus structure, but pay them out on, for example, a quarterly basis.

I can't really say that just because there would be more 14s than 13s that that would necessarily result in a more even flow of production, other than the fact that if you're a 14 and you're maintaining at least a fully successful rate and at least putting out more files as a 14, and maybe this is where you were going, then the files you were putting out as a 13, and you have that many more 14s, then obviously, the cumulative number of files that are going to be disposed of are going to be more on a daily, weekly, and monthly basis, let alone a yearly basis than they would be.

But the simple fact that there may be more 14s than 13s I don't think would necessarily correlate to having files being sent out on a more even basis.

As one other point, just to follow-up on a couple of things that I think John as well as Chris said, obviously, this committee needs to focus as they have on the bottom lines and that is getting files out as quickly as possible at a reasonable cost. I would hope that the committee wouldn't lose sight of the fact that on a number of major areas that are important to this committee, including expansion into additional space, including a PTO corporate -- including a corporatization bill, the reprogramming and other legislation that has been on Capitol Hill.

I think it is important to remind the committee that into NTEU 245 as well as NTEU has worked closely with INTA to support a corporatization bill, has worked closely with the office to follow-up with what Todd and Bob said to support a move to a new facility as opposed to, example, one of the other unions in the PTO which is literally spending hundreds if not thousands of hours in collective bargaining over issues that we could have been spending in collective bargaining as opposed to examining applications which is really what you want us to do.

So, there have been a number of areas where I would want the committee to be aware of for those who aren't of the way partnership has worked and has worked well between our union and Bob and Bruce, and in the future, Todd.

MR. KRUGMAN: Thanks Howard. Susan?

MS. UPTON-DOUGLASS: I was just going to say that when we discussed this last at the NTEU/SPTO subcommittee meeting, there was a very strong level of support for your proposal and I think we just look at the 19 people who left and the 49 people who came on and we say you know, this -- and the productivity comparisons and we say we'd rather keep the people who are doing a good job.

MR. KRUGMAN: Yes, I haven't heard anything in a negative way to your proposal. Anybody that wants to say anything, we're going to pose the question to Bob Anderson because no one has sort of taken the other view. We'll pose that to him later, but it sounds like a consensus of the committee that this is something that we'd like to see for the reasons you've stated.

Is there anybody of the observers that would like to comment about this?

MR. WILLIAMS: I would.

MR. KRUGMAN: Go. This is Ron Williams.

MR. WILLIAMS: My name is Ron Williams and I'm a senior Trademark administrator. I'm not speaking for Bob Anderson, but I don't think -- I think it's safe to say that the office is not opposed to the GS-14 as a grade. It's just a matter of trying to work out the mechanics of how we're going to get there.

As for the mentoring, the training aspect of it, the 14s, when it was originally, I think, created, they were going to do mentoring and because we've been hiring so many people at such a fast rate that we've run out of GS-14s that we feel confident to have training and we don't think it's in the office's best interest to have anyone training who we don't feel is qualified and ready to do that.

So, we've taken it out of the GS-14 announcement simply because we're bringing in -- we're going to hire 150 people within a 12-month period and we just don't have enough GS-14s to train them the way that we've been doing it.

MR. KRUGMAN: Ron, what's the working grade of the managing attorneys? 15?

MR. WILLIAMS: Yes. We can talk about that too.

(Laughter.)

REPORT ON CUSTOMER SURVEY

MS. MARSH: The next topic on the agenda is quality of work products from the Trademark Office and we had a customer survey and we wanted to give you the results of that survey.

This is Mr. Mullens. Do I have that correct? Greg Mullens. He's going to give you a briefing on the findings.

MR. MULLENS: Actually, I'm just going to introduce Brian Usilaner from Westat, which is the contracting company that conducted our customer survey this year. For some of you, you might remember, this is the third time we've done it. We did it in '95, '96, and '98, and I think some of the data this time is going to be very interesting to look at.

So, I'll turn the meeting over to Brian to guide you through the briefing.

MR. USILANER: Good morning everyone. We conducted the customer survey. We do a lot of customer surveys, both in the private and public sectors and we conducted this survey for the Patent and Trademark Office. Obviously, there's a lot of detail, a lot of analysis that went behind the data.

What I would like to do this morning is just give you an overview and highlight of some of the results and if you have any questions, or let's keep this as informal as possible, if you don't mind. So, interrupt me as I go along, if you have any questions or clarifications. You have all of the slides that I'm going to use in the booklet. I'm not going to use all of them, but you have all of them in the booklet.

Okay, first of all, the survey was conducted, it was a mail-out I guess of about 1,000 was the sample taken, and the survey was conducted between June and July of this year. There were very few changes that we made on the survey this year. We're going to revisit that next year, but for this year, we made minor changes to the survey so we could keep the continuity of the survey instrument.

We added some questions about problem resolution. We added some questions about value as opposed to just cost and a few questions about service quality. One thing I want to point out to you, you'll see that when the data is broken out as I go through is there is a cadre of questions on the survey that deal with performance against standards, standards that the Trademark Office had set and the respondent is asked to assess performance against that standard.

The scale on those questions had no neutral point. Either two points of dissatisfaction and three points of satisfaction.

Then there was a cadre of questions dealing with the examination process itself. That did have a neutral point so there are two points of dissatisfaction, two points of satisfaction. The only reason I'm pointing it out to you is you obviously can't combine and compare those two sets of scales.

Greg, next slide please. This was a mail-out survey followed up by a post card and a second mailing and for mailing surveys, this was a high response rate. We usually get about 40 or some percent. This was 52 percent response rate and as I said, that's excellent for this type of survey.

We asked at the beginning of the survey questions about demographics so that we can get a profile of the respondents. As you can see from this chart, the majority of the respondents were from law firms -- can everyone see this? You have it in your book -- 78 percent. It was exactly the same as the last survey in '96 and there were some small, minor changes in between.

Large businesses dropped a little bit and I think small businesses went up a little bit. But those are small changes. The majority of the respondents are from law firms. In addition, most of them had applied for a trademark within the last 12 months, so it's fresh on their mind and that's what they're referencing.

In addition, we asked questions about -- and this is important to keep in mind -- frequency of contact and whether you're a continuous customer or not. Obviously, from law firm's perspective, there is a lot of interaction throughout the year between the "customer" and Trademark employees.

The reason I'm pointing that out, that has a lot of ramifications when you asked the respondent to assess service quality. The whole concept of the service quality area is called moments of truth. Every time you have an interaction, obviously, you're making an impression, either positive or negative on your customer. So, with not having a one focal point, for example, some companies have just a customer service center, and it's only the center that interacts with the customer.

All the other employees are "behind the curtain." They don't do any interaction. That's much more limited in terms of the contact and the frequency of contact. This has a lot of contact and that's important and these people are continuous customers that deal with you over time.

We broke the data out a few ways here, just for summary purposes here. We just took first an overview of the survey and said of all the questions on the survey, what questions have the most positive satisfaction response rate? As you can see on this chart, those questions in terms of the process itself, the most positive questions deal with the application submission process.

I know you don't have a formal submission form, but obviously, these are law firms that do it over and over again and obviously, they're very comfortable with the submission process. I guess I'm talking to the choir when I say they. It could be you that very well might have filled out the survey.

In terms of questions pertaining to performance against standards, there's some interesting points on this slide. Courtesy is extremely high and that's important. In fact, this is what I said -- as I said, I do a lot of surveys both in the private and public sectors so I have a lot of benchmark data.

This is extremely high for this type of organization where you have that number of people dealing with the "customer." That's a very high rate and as I said, with frequency of contact and the number of people, you wouldn't expect courtesy to be that high.

In addition, clear written communications which is a quality indicator for the examining attorney is high as well. That's at 81 percent and something that's very important to any customer is the whole phone call cycle, the whole phone call process. Returning phone calls promptly was very high as well.

In terms of the questions that the respondents were least satisfied with, the lowest level of satisfaction, in terms of the process, they have to do with handling of delays, handling of mistakes, and you'll hear a term I'll be using frequently this morning, "the whole problem resolution process," how the office recovers when there is a problem, whether it be a mistake or a delay of some type, and these are very, very low levels of satisfaction.

In terms of the least satisfied in terms of those questions pertaining to performance against standard, there are two that stand out here. That's filing receipts within 14 days and not surprising, because of that, first communication within 3 months, and then the whole process of meeting that 13-month goal is low as well. So, you can see there is a significant timeliness issue, especially in the up-front part.

The second cut of the data that we looked at was -- this was just, as I said, a global look at the most positive and the least positive. The second cut was simply to compare the questions in terms of percent satisfied from '96 to '98, and look at the areas that had the major improvements, and then look at the areas that had the major declines.

In terms of improvements, not surprising, the cost -- their satisfaction since I guess cost has not been raised, the respondents were happy about that and therefore happy about the cost.

I said that we know this from us being customers in other aspects of the service industry. The whole phone call operation is important, directed when you call promptly to the proper office or person was also a high level of improvement over '96. It went up 10 percent.

Other areas, mailing the certificate of registration within seven days, clear written communication, which I said is an indicator of examiner attorney quality was high as well and some other areas were also in the 6 percent, 5 percent which are statistically significant.

Just from a rough rule of thumb, you're looking at the lowest level that you would be satisfied with in terms of percent satisfaction is about 60 percent. That's the lower threshold in terms of benchmarking with other organizations of this type. Obviously, you would like to get in the 70s and 80s, especially those questions that have the three levels of satisfaction. So, obviously, anything below 60 percent is a significant problem area.

In terms of the major declines -- and I want to point something out in terms of the construct of the survey instrument -- one of the changes we did make was in the last survey, they asked -- there were three questions about problems: handling of problems, handling of mistakes, and courtesy in handling your problem.

There was not a sort question, meaning everybody responded those questions, supposedly, whether they had a problem or not. This year, we had a sort question. So, you were first asked if you had a problem and if you did, you answered these questions pertaining to the problem resolution process. If you didn't, you skip these questions.

So, again, not surprising from the data we already saw about the lower levels of satisfaction, you get the whole problem resolution process that we asked last time, including courtesy dealing with the problem went down. So, handling of delays and handling of mistakes, a significant drop even from '96 and some of it can obviously be explained away because you had some people that didn't have problems responding.

But nevertheless, if you look at the level of satisfaction, that's quite unacceptable.

MS. DARE: Can I ask a question? About the courtesy point, I noticed on your overview what respondents were most satisfied with. It says 81 percent or 85 percent felt that they were treated with courtesy each time there was a contact, but then clearly, when there was a problem, that's not -- you know, you got a very different high dissatisfaction.

MR. USILANER: Not a high. It's still 66 percent, but obviously, these people that have problems, a little more upset, and a little more negative of the courtesy. Still, that overall question pertaining to all of the interaction whether you had a problem or not is quite acceptable at 85 percent. It's just that this has dropped a little bit and one of the reasons why it dropped in comparison was, as I said, there were some people that didn't have problems that responded to it last time.

Obviously, this is something to look at and we'll get back to that point when I talk about my recommendations because it has to do, as you're pointing out, is that interaction and the "problem resolution process."

We asked a question that's a good anchor question to do analysis on; at the end of the survey all in all how satisfied are you with the trademark process? As you can see from the data, comparing it to 1996, there wasn't any change to speak of. Now, there are a few things to point out on this chart.

First of all, as I said, 60 percent is acceptable. It's just above 60 percent. The second point, though, is that your people that are dissatisfied is low. The rule of thumb there is about 25 percent. These are just rules of thumb from, as I said, from benchmark data that I've dealt with; 25 percent.

So, you obviously, in order to improve that level of satisfaction, you're really focusing on the people that are in the neutral category, which is a legitimate response. There are people that see some things that are good, they see some things that are poor overall, they're in the middle.

So, it's a lot easier to move those people to satisfy than the people that are on the negative side. Now, you've got to remember again when you look at this data is that we're talking about a "monopoly" here. You don't get much high levels of dissatisfaction in the private sector because the customer has the choice of going elsewhere.

So, you have an automatic filter of those people that are dissatisfied. They'll do business with other people. Here, obviously, you don't have that case. Your customers, whether they're satisfied or dissatisfied have to stay around. So, that's the good news on this chart. We'd like to see more movement and we have a theory of why there are some large percents in the neutral category.

I told you that that question is good for analysis. There is an analysis that's done called "key driver analysis," and all that means is statistically taking a look at all the questions and then statistically determining which questions have the greatest impact on overall satisfaction; meaning if you focused on these questions and made an improvement on these questions, you'd get the greatest bang for your buck in terms of improvements in overall satisfaction.

You don't have to ask questions anymore that we eliminated in terms of importance because by the way the respondent is responding to the questions, they are telling you which questions are more important than others in terms of those questions' relationship to the overall satisfaction question.

So, as I said, this helps for any organization to focus on those vital few issues that are important to the customer because they have the greatest impact on overall satisfaction, and we have them in two parts here: those questions pertaining to performance against standards, and as you can see here, courtesy is important, not surprising that clear written communication, which is a quality measure is important.

The whole phone call operation is important, directing calls, and then the timeliness of the whole cycle, meeting that 13-month goal is important as well as the filing receipts, meeting the 14-days in filing receipts.

On the other side, in terms of the process, if you'll look here, you'll see some questions pertaining to "service quality," flexibility, quality of the feedback provided, committed to providing the best possible service. This morning, we don't have the time to get into a lot of the detail, but those are being driven by the negative reaction about problem resolution, as we'll see.

It's about -- I think it's about 58 percent, if I remember, of the respondents claimed they had a problem. So, they answered -- 58 percent of the respondents claimed they had a problem of some type or another, and that's driving some of those, the flexibility committed to service down.

In addition, the length of the application product process, and the issuance of the product in a timely manner, which is the end of the process, is also important, which had lower levels of satisfaction. So, you can rightly see from this the areas that need to be concentrated on to improve overall satisfaction.

I put an example here, just to show you the impact here. So, a key driver was meeting that 14-day of filing receipts. So, we did a cross-tabulation here. We took a look at those people that were satisfied in terms of the performance in meeting that 14-day target and those people that were satisfied, their overall level of satisfaction was 82 percent.

Those people that were dissatisfied, which were the majority, in terms of meeting that 14-day goal, their overall level of satisfaction was only 50 percent, and you can see the same thing about the view about commitment to service as well. Register ability of Trademark within 13 months, so it's the same dynamic gap between those that are dissatisfied and satisfied with these key drivers in terms of timeliness.

There were a high level -- again, from past experience -- a high level of write-ins. I think something like 65 percent of the respondents wrote in one or more comments. That is high. So, obviously, the respondents want to be heard, number one; and number two, have something to say; and number three, want to make sure that message is being heard in terms of improvements that they would like to see.

So, we did an analysis of the open-ended question and you have a lot of detail in your booklet here for each one of the questions, what pleased you the most, what displeased you, what changes you would like to make, what new products and services, and so on, and you have the detail in terms of broad areas in front of you, in terms of we categorized, did a content analysis of each of the write-ins, and we provided Bob with copies of all the write-ins, cleaned up for language to protect the guilty or innocent, or what have you.

Let me give you an overview summary of what we gleaned from analyzing these large number of write-ins. First of all, on the positive side, they believe that most examining attorneys are committed, responsive, helpful, and courteous.

The second bullet, there were a lot of write-ins, and this is my language, when the examining attorney was what I call proactive; a very important point -- proactive, not waiting for the "customer." I keep on saying "customer," but there were some write-ins, darn it, I'm not a customer. I have to do business here.

Anyway, the customer, instead of initiating the call, got the call from the examining attorney dealing with things such as expediting the process, solving minor problems, suggesting ways to simplify the process, and shortening the time cycle, et cetera; a lot of examples of that type.

Most comments or I think over half -- in fact, I'm positive over half the comments involve timeliness issues of one type or another, a lot of complaints about the time it takes, as we've already seen, to get filing receipts. The perception of a long time of getting the application assigned to the examining attorney, first action, getting certificates of registrations, a lot of comments about timeliness throughout the process.

Now, that's a perception obviously, because as we know -- because most of the law firms do a lot of business here. They're not reluctant to pick up the phone to find out if the application has been assigned or not.

Another important point is when things go smoothly, take a look at the people that had no problems. Their overall level of satisfaction, we didn't show this, was something like 90 percent. Those that answered that question did not have a problem. Their level of satisfaction was 90 percent.

When there is a bump in the road, the perception is it's hard to get back on track. It's hard, it's difficult to fix because any fix takes time, and the time further delays the process. It's not simply a quick fix and so you have a lot of comments of that type. Lost files, lost drawings, any errors on the filing receipts, any of those type of errors, the perception is it's an error. It could be a minor error. Whatever the error is, it's hard to get the error fixed. It's not simply picking up the phone and correcting over the phone. That's the perception.

The mail process was used. These were the terms that you all use. Mail process is viewed as causing many errors. I already said correction requests delay the process and there were positive comments about the status line.

Now, let's take a look at overall conclusions in terms of the strengths. Overall courtesy it is a very high level. We talked about examination quality in terms of the response. The indicator of service quality is high. The application submission process is high, and there are improvements that are being made in the whole phone call process, especially returning calls promptly and promptly directing the calls to the proper person or office.

I mention here, though, that there are several key aspects of customer service that are acceptable levels and are showing a positive trend; returning the calls within one day, the status information, accurate responses to questions, and expert advice about the process.

On the negative side, in terms of areas for improvement, the entire problem resolution process got extremely low marks on all the questions. As I already mentioned to you -- I didn't mention to you, I'm sorry, that those people that had a problem, and then asking the question if the problem was fixed okay, their level of satisfaction went up, I think close to 90 as well.

So recovery is important. The perception of recovery, if it's done well, can have a significant impact on overall satisfaction. Then, the problems that we talked about in terms of timeliness, all these areas represent and remain a serious problem, and some of them are important, the overall process time and some of the standards, the filing receipts and so on are important because they're key drivers to overall satisfaction.

Okay, I already mentioned timeliness is a key driver, and then the flexibility in addressing needs, assistance at a time convenient to you -- we did some analysis of that. Those are lower levels in terms of service quality and what's happening is those people that have problems have a negative view of this type of thing; flexibility, the whole fix.

So, it's having a spill-over for those 58 percent. Of the people that had a problem, it's dragging down several of the questions about service quality; not surprising, but it's dragging it down.

I mentioned most of the areas -- problems or delays because of timeliness issues. We did another analysis to show the significant impact when you handle delays correctly on overall satisfaction as well.

Then the final chart is some of our recommendations, what I call the vital few. Filing receipts is a key driver, based on the write-ins and based on the responses to the questions on the survey, both errors and in terms of meeting that 14-day goal, there's a high level of dissatisfaction.

Because of the high number of incidents identified, especially in the write-ins that give you a lot of richness to the questions on the survey, a lot of things that are lost, missing papers, lost files, fees, specimens, drawings, and so on, that's usually an indicator that there are opportunities for internal process improvements, obviously. Now, this has ramifications because it further delays the process. We've already seen that timeliness is a key driver as well.

Finally, whether there is a formal process or not, the way problems are handled needs to be improved and there are a lot of good data and information, both in the public sector and the private sectors, because organizations are starting to realize the importance of the whole problem resolution, the problem recovery process.

So, there are significant opportunities for improvement of how to handle them and I think some of it, as we had a discussion with the managers and employees, there is a lot has to do with the term I use, "managing expectations." When you have goals of this type that can't be met, you're setting up a false level of expectation for the customer.

The more realistic that you give the customer expectations of what they can expect in terms of both the timeliness and in terms of the service, usually, it has a significant impact on the ratings of that service. So, if you go into a restaurant and are told that it's an hour wait, what the restaurant has done is managed the expectation. So, restaurants plan for that and know that they can beat it and you're delighted when you only had to wait 45 or 50 minutes.

There are many examples in the service sector of managing expectations. If I think that as soon as I get the filing receipt that my application is going to be assigned, then I'm going to have a negative view if I find out that it's not going to be assigned.

We can go into a lot of this -- we had a lot of discussion about that, and I said priority should be given to handling delays, especially in the problem resolution process because of the importance of timeliness.

Now, this gives you an overview, and obviously, the discussion is going on now of what to do with this data information and what improvement actions can and should be taken. With that overview, I'll be glad to try to respond to any questions. Yes, sir?

MR. KRUGMAN: John?

MR. RYNKIEWICZ: John Rynkiewicz. On slide TE30, on the open-ended comments, you mention on the last bullet timeliness and examining attorney competency. Throughout this whole study, quality is highly rated and it's listed as a strength. Is that just in that pocket of the open-ended --

MR. USILANER: Yes, if you turn to 31, of those that responded to Question 21 -- and we lumped these. There were various aspects of staff competence. We didn't break them down between examining attorney and clerical. Some were hard to break down that way. I guess you could. There was only 26 percent of what they disliked.

There were comments, obviously, not surprising that there's a variation in the "quality level" of examining attorneys. Some -- you know, some respondents praised and some criticized, but on the write-ins, those were the two areas most identified. They still were not drawing down the overall question about examining quality.

MS. UPTON-DOUGLASS: Now, I was one of the people who was given a survey and I have to say that I had a lot of problems doing it because first of all, we file -- or I personally file, I would say, thousands of applications a year and when you're dealing with that kind of volume, I mean, you're not going to condemn the entire office.

I mean, there's a patch of things that aren't right and I think we know them. We talk about them at these meetings and at our association meetings, but it was very difficult to answer the questions the way that they were posed. For example, it would have been helpful to say overall, this quality of examination in terms of statutory objections, you know, were high or low, or whatever, but it would have been better to say maybe 50 percent are good quality, or 25 percent are of high quality, or 75 percent are of high quality.

But to have these survey questions, I found that I couldn't answer them because some experiences were good and some were not. So, I would pay particular attention to the write-in comments and I spent a lot of time, for one, with the write-in comments because you couldn't fairly answer the way your questions were phrased.

MR. USILANER: Right. Greg, do you want to respond to that?

MR. MULLENS: Go ahead.

MR. USILANER: Because a decision was made and that's why Greg is author this year, that this was a survey that was designed originally by the Census Bureau and some of the questions, the wording of the questions were not appropriate, et cetera. We set up an 800 number for people to call us whether they wanted clarification.

We got a lot of calls and a lot of those calls pertained to the survey itself. We catalogued those, we got a lot of comments, and we're going to make recommendations to PTO about improvements in the whole survey design next year. Obviously, you want to hold on to certain questions so that you get a level of -- a trending level. You don't want to change all of them, but some.

It was obvious that the respondent had a difficult time; difficult, as you can see, based on I read every one of the write-ins, as I said before, about "staff competence," well, there was no break-out in terms of the questions on the survey between the technical support people and the examining attorney people, as an example.

As you point out, you have a lot of interaction in terms of throughout the year, so when you're asking the questions, are you summing all that up, are you responding to the latest, et cetera?

MS. UPTON-DOUGLASS: Yes, and I think that explains the comment that John Rynkiewicz raised as well, which is to say -- I mean, there are a lot of problems with quality and I wouldn't say -- I don't think people can go away from this survey and say quality is 82 percent, quality at the Trademark Office on substantive issues is stellar, but some of the people are consistently very good.

MR. USILANER: Right.

MS. UPTON-DOUGLASS: And some need help, and you know, this kind of survey, really it's very hard to get at this kind of data and that's inherent in any kind of survey.

MR. USILANER: I can't make a commitment. All we can do is recommend. That's why Greg --

MR. MULLENS: I think next year when we look at the survey, we are going to get into some redesign this time for some of the questions.

MR. USILANER: Because the 800 number is really helpful to us because people were frustrated in terms of responding to some of the questions and we wrote down each of these so we have some data this time. We're not just flying blind in terms of changing the design of some of the questions. Your point is well taken.

MS. DARE: You know, trying to draw on some of Susan's comments and just some real recommendations, I think especially for practitioners, usually, in law firms who are doing more than 100 applications in a year, I would really want several different metrics of you know, are they satisfied with 75 percent and dissatisfied with 25? I'd be very interested to see them break down by percentages.

MR. USILANER: Say this again, percentages?

MS. DARE: For example, Susan was saying that maybe 10 percent of her applications -- I'm making up that number -- are very good and she's extremely satisfied.

MR. USILANER: Oh, I see.

MS. DARE: And with 75, she's very middle of the road, and then with another 15 percent at the bottom, she thinks those are just terrible; there are mistakes, there are gross errors, or whatever. To have an initial metric like that from people who are doing 100 or more, or 50 or more applications a year, I think you'd get some very valuable trending data.

I'd be interested not just in a practitioner's satisfaction at those different levels, but also some kind of trending by examiner if the practitioner had enough experience to see that. You know, I feel that I'm extremely satisfied with 15 percent of the examiners that I dealt with and that they're very consistent and very good.

Then, there's a middle cohort and I feel somewhat differently about them, and then there's a bottom cohort where I get either very patchy results or I'm consistently dissatisfied with those. So, I think those are some metrics from people who are doing large volumes and that's invaluable quality data.

MR. USILANER: Right, yes, because there are really two types, two major types of customer survey instruments that are used. One is called, for lack of a better term, an overall survey. Say you had a lot of interactions. Step back now and try to answer these overall.

The other type of survey that we're seeing more and more of is what I call a transaction survey. I just took my care in for service. Now, that last transaction, that last server fresh in my mind, you get a mini-survey or a mini phone call survey of some type asking about that transaction. You call the phone company to come out for repair, what have you, you get a mini-survey about that transaction.

So, in the design, we have to be careful here is that we want to get an overall reaction, your -- when we take into consideration, that's a good idea, as well as some type of transaction should be thought of. There are two separate issues, especially in light of many of you having more than one application.

MR. KRUGMAN: Anything else?

MR. SIDOTI: I'm thinking it might be worthwhile when -- Chris Sidoti, ABA. It might be worthwhile if randomly, when somebody files an application, they're given back with their filing receipt sort of -- you know like when you stay at a hotel, how did you like your stay kind of thing -- just randomly given out and of course, there are vagaries, if they get a refusal and all that and that has to be taken into account.

But it might be a good way to kind of monitor how the process is going across the board so that when it's complete, either there's a final refusal and you can weigh that in or there's a registration issued. Then, that can be submitted and that data might be very worthwhile to track to see how on a given kind of situation, very objectively, how the customer views it.

MR. USILANER: That's the transaction I was talking about, right. So, that's something to consider, obviously, as another type of survey, as you point out, on a sampling basis, those that have gone through that have had their application approved, or what have you, registered, get a mini-survey -- not as long as this one, just a handful of questions.

MR. KRUGMAN: Are you talking about like an ongoing basis?

MR. SIDOTI: For a period of time, right. You know, just sort of take a post-hole for like maybe a month period of everything that's filed and then out of that grouping, select a universe objectively and just see what the -- you know, it may take a year to get these results, but I think it might be very worthwhile and it might be more meaningful data.

MR. RYNKIEWICZ: John Rynkiewicz. A few years ago, weren't there some kind of -- at the end of the form paragraph, there was a send some comments? Was that to your group, Chuck.

MR. CONDRO: That was to us. Chuck Condro, Trademark Quality Review. There was a form paragraph at the end of each office action soliciting comments and we got some comments for a few months and then they basically died away and we didn't get anything. It was good for a couple of months and then it just faded.

Then, every once in a while, a straggler letter of complaint would come in and it would have nothing to do with us. We'd have to send it to somebody else to straighten out whatever the problem was, and what became of it. So, from that standpoint, it only lasted for about a couple of months -- two or three months in which there were just people that would just come and give an overview of what they saw and after that, there just were no more responses.

So, they cut out -- I think the operation cut out the form paragraph at the end of the letter.

MR. SACOFF: Bob Sacoff, for IIPA. I think the survey is a very worthwhile and interesting piece of work. I think it has generated a lot of very useful data. Sort of from an overall standpoint, I just wanted to make the observation that in my view, one of the things that you've mentioned is exactly 100 percent on point; namely, the problem resolution process is an area of difficulty.

It's an area of difficulty in any organization no matter what you do, but I think it's extremely important for the committee and for the administration to recognize that as an area that is generating a disproportionately large amount of angst among the clients, customers, forced users, whatever you want to call them and that perhaps a focused task force approach-type effort towards addressing those types of issues would be very productive in terms of paying back and in terms of a much higher level of efficiency in user satisfaction.

MR. USILANER: Yes, because you're paying the price two or three ways. One, as you saw here, when you have to "fix" a problem or respond about a delay, it's taking away from the productivity, especially, of the examining attorney. To "fix" things is rework, which also has an impact on the efficiency of the whole operation, and you saw, as I said, in the analysis that we did, for those people that did have a problem and that were satisfied with the way it was handled, their overall level of satisfaction shot right up.

I think it was equal to, if I remember, or a little higher than those people that didn't have a problem.

MR. SACOFF: I'd even like to add my personal observations. The times when I have had personally the highest degree of frustration with the PTO process is when there's a problem that, you know, in the ideal world would seem like a relatively fixable, easy to fix problem, but you try time and time again, it's when you try to get a filing receipt corrected and you mail it in and it comes back wrong the second time, and you mail it in a third time, and it comes back wrong the third time.

Or, when you've got a problem with a filer, finding a filer, fixing something in a file, and you get bounced from person to person and nobody knows and nobody can find -- you know, it brings back all the worst Kafkaesque nightmares of a bureaucracy. So, you know, to return to a more rational tone --

MR. USILANER: And that's nailing down some of those service quality questions of those 58 percent, in terms of flexibility, meeting my needs, generally committed to service and so on. When we analyze that, those people that are dissatisfied with the way a problem is handled had a very low level of satisfaction with those types of service quality issues.

You can tell from the write-ins that you have more than just a few vignettes of the type that you're talking about when things, or my words, off track, the difficulty in getting it back in track.

Okay, thank you very much for your time and attention.

MR. KRUGMAN: It's about five until twelve, so, Sharon.

MS. MARSH: Do we want to hear Chuck's report before lunch?

MR. KRUGMAN: Absolutely. Chuck, can we hear your report before lunch?

OFFICE OF TRADEMARK QUALITY REVIEW REPORT

MR. CONDRO: I'll try to make it brief and concise. I'm don't want to put a wet blanket on this rosy summary here so I'll try to present the state of quality as we see it now and hope that it's not too inconsistent with what you just heard.

Let me give you a quick historical background. Trademark Quality Review has been in existence since December of '88 and we report to the Deputy Commissioner and the reason for that is because the Department believes that there's an inherent contradiction between quality and quantity and felt that there should be an independent office that reports to the Deputy Commissioner rather than to the Assistant Commissioner for Trademarks, and the same situation exists on the patent side. The Patent Quality Review also reports to the Commissioner's Office.

The primary objective of the office is to provide a mechanism to alert the Trademark examining operation to recurring problems in order to improve public confidence in the examination process and the validity of issued registrations; specific objectives, detecting trends away from proper examination practice and procedure so that corrective action can be taken, provide feedback to management in the examining core for both educational and corrective action.

We compile, analyze and disseminate information regarding the current state of quality and we provide statistical summaries of the results of the quality review process to management. Particular attention is paid to what we commonly refer to as clear error and examination, and that's errors that effect registrability. So, it's an interchangeable term. You'll hear me say errors that effect registrability or clear error; those are interchangeable.

We look at an application in four specific areas; the application of substantive statutory criteria for registrability; in other words, errors of substance. We do a sub-sample search for confusing or similar marks. We're the only organization within Trademarks that not only reviews an application, but also searches. We duplicate -- well, we don't duplicate. We search an application as though taking into consideration we're not using examiner search strategy. We search afresh.

We review each search strategy to make sure that the search strategy is correct and then we review it to make sure that the examiner has adhered to approved examination practice and procedure.

Now, in these four areas, what we do is we identify and then pinpoint the most common place of recurring errors. Now, when I mean pinpoint, I mean by law office. We do not and have resisted, and I will continue to resist any attempt to try to keep track of which examiners are making which mistakes because that's going to -- you know, what we try to do is we try to pinpoint the error.

We try to find out number one, what errors are occurring most often and in what law office they're occurring. The file selection, it's a 4 percent, computer-generated random sample of examined applications in all stages of pendency, including abandoned and registered applications.

Now, there was some discussion a few years ago of why look at abandoned and registered applications? You can't really do much about those. But the statisticians believe that in order to have a statistically valid sample, you also had to include in your sample applications that had abandoned and registered. Also, the Department Inspector General felt very strongly that abandoned and registered applications should also be looked at.

A follow-up review is conducted to make sure that whatever the Trademark examining operation agreed to correct is corrected. Now, in our first few years, we didn't do that. The Inspector General again came in and indicated to us that how do you know that these errors are being corrected? They've said they're going to correct a particular file, how do you know that? I said we don't, and they said well, we want you to pull that file after they've said they're going to correct it to make sure that the file has been corrected.

Let me give you some current data as far as the current state of quality and I'll try to give you these few percentages and try to match them up with applications because I don't think the percentages I give you or the error rates in the three or four categories are going to be that helpful unless I give you also how that translates into applications.

The error rate for 1998, in our opinion, was 3.8 percent which was a slight drop from the fiscal '97 figure of 4.2 percent. Now, that translates into about over 7,000 applications in which there was an error that effected or could effect registrability. Now, the thing that you have to be concerned about about that error rate is okay, so, fiscal '98 appears overall that the operation did a better job in fiscal '98 than it did in '97.

However, in the last six months, the error rate is 4.3 percent, which means in the last six months, the error rate has exceeded the error rate of fiscal '97. So, it's a warning sign that in the substantive statutory criteria area, the error rate is beginning to rise; not dramatically, but it's beginning to rise.

Missed references -- for 1998, the missed reference rate was 1.8 percent, which translates to about 3,500 applications that we believe the examiner missed a reference which is a significant drop from fiscal '97, which was probably one of the worst years since we -- I think the second worst year since we've been in operation. In fiscal '97, they had almost a 3 percent error rate in missed references.

The rate of error in this category, missed references, has remained pretty steady since mid-year. It hasn't risen, it hasn't gone down. It's maintained almost close to 2 percent throughout the last six months.

Search strategies, historically, that's been the best quality of examination measure. The TMEO has consistently been below 2 percent in the way they frame their search strategies to look for missed references. This year, the error rate was 1.2 percent, which means that in 2,100 applications that we looked at, we felt that the search strategy was deficient, and whenever the search strategy is deficient, then we -- in those particular cases, we research the mark because the deficient search strategy could have resulted in a missed reference.

However, even in that area, which has been historically the best area, the last two months -- last two or three months, the error rate is close to 2 percent, so in the search strategy area, the error rate is beginning to climb.

In practice and procedure, the error rate, they finished 1998 with a 5 percent error rate, which was a significant drop from fiscal '97, which was 6.3 percent and in fact, the 5 percent error rate has held pretty consistent throughout the year.

So, in the clear error rate, even though it's 3.8 percent, it's on the rise. The last six months was 4.3 percent so it's a warning that something is going on and search strategies, which have been historically -- they've been below 2 percent for 5, 6, maybe 7 years, a long time. All of a sudden, the last couple of months, which is really still too short a period of time, but in the last couple of months, the error rate and the search strategy area has risen.

Now, the follow-up review that we do, in 1995, they only corrected 86 percent of what they said they were going to correct, but as each year has gone on, they've gotten much better at that. In 1995, it was 86 percent. In 1996, they corrected 89 percent; 1997, they corrected 92 percent; and in 1998, they corrected 94 percent of those applications that they said they were going to correct, they corrected and we pull -- like I said, we re-pull the application and we check to make sure that the correction has been made.

Some historical perspective on the numbers, the fiscal year '98 was only the second year since we've been in operation that the rate of clear error, which is the most important category really because it includes the substance -- the substantive mistakes -- fell below 4 percent, even though, like I said, it's been up the last six months.

The only other year that it fell below 4 percent was 1996 when the operation finished at a 3.5 percent error rate. That comes to search strategies fiscal year '98 was only the third year since we've been in operation that the rate of misreference fell below 2 percent. The other two years were 1995 and 1996 when the error rate, number of missed references were 1.8 percent and 1.6 percent respectively.

The operation has maintained a rate of deficient search strategies below 2 percent since 1994 and the error rate in the area of practice and procedure. This year's 5 percent rate is the lowest error rate since we've been monitoring practice and procedure.

I guess if I would just conclude by saying that the state of quality is mixed. Overall, it may be better than last year, but there are some warning signs out there and it could be due strictly to the fact you've got a lot of new people. The last five or six months, there seems to be a rise in the number of substantive mistakes and it seems to be a rise in the number of strategies that are deficient.

MR. KRUGMAN: Chuck, when you're reviewing these applications, do you count a clear error? Do you look at other statutory grounds like --

MR. CONDRO: Everything. 2D, 2E.

MR. KRUGMAN: What about an error of commission? Would you count a descriptiveness refusal an examiner made that you thought was not well taken as an error?

MR. CONDRO: Okay, yes. Here's how we handled those situations where an examiner has a 2E-1 refusal or a 2D refusal that we kind of don't agree with. We give them a little bit more latitude, frankly, when they've made a 2D-1 refusal, then they withdraw it versus let's say they didn't make it at all and we felt it was 2E-1. Now, if we think it's really wrong-headed, then it goes down. Then, we'll send over a finding, yes.

But that's not really where the -- the only area where we see -- the two areas that we see where they make refusals -- I mean, they've made 2D refusals that I wouldn't make and they've made 2E-1 refusals that I wouldn't make, but that's not as frequent as the one area that we see a lot of is in the service mark area with specimens.

What they'll do is let's say it's an arbitrary. I don't want set forth a mark, but let's say you have an arbitrary word and then you have two or three more words that are descriptive, or generic, what have you, and they'll ask for a disclaimer of the descriptive term, which describes the service, and the specimens as far as the format is fine.

They'll come back and they'll ask for specimens because the specimens don't set forth the nature of the service. Yet, they've just asked for a disclaimer of the descriptive term so they're forcing the applicant to send in specimens, which is a good chance he or she doesn't have.

This happens with a fair amount of frequency where in the Trademark area, it's not as big a problem. In the service market area, we think they're too adversarial, that they ask for specimens and service mark situations when they don't need to.

MR. KRUGMAN: Does anybody have questions?

MR. FELDMAN: At what stage in the process are you doing this, because -- is this where the situation is often after publication?

MR. CONDRO: It's everything.

MR. FELDMAN: And I get the call from the examiner very sheepishly that I apologize I have to call you, but there's this word in the description of goods that really isn't right and --

MR. CONDRO: Okay, we look at the application in all stages of pendency. It could be at the first action, second action, final, publication. This computer spits out these serial numbers and those are the only ones we're permitted to look at. We don't look at any other case other than that, and it can be in all stages of prosecution, including abandons and registered.

So, I don't know if I've answered your question, but it's -- there's no part of prosecution that's not subject to a review. It really depends on whether the computer spits out the number. If the computer gives us the number, which we get every two weeks, we're going to try to find the files, which is not easy. It's very difficult to find these files.

MR. FELDMAN: But aren't you also creating situations where I've survived an opposition period and then because of a formality, such as a description of goods, I have to get published again?

MR. CONDRO: Well, it's possible that a file could be two, three, and four years old. That's rare, I mean, to find a substantive mistake. We might find a deficient search strategy, but it's very rare to find a substantive mistake after a file has been through a whole long period of time.

But in theory, yes. In theory, a file could come along that you've been working on for several years. It's had extensions of time and so forth. We get it and all of a sudden, we go this thing is descriptive. Yes, in theory, that can happen. It doesn't happen that often, but it can happen in theory, yes, and you're right. Then, it's embarrassing to everybody.

But the office also has -- well, go ahead. I don't know what else to say to you, except that in theory, that can happen. It has happened in the past, but that's not normal because it seems that most of the files that we get that the computer spits out are cases that have had first action, and second actions and that are published.

I would say a very small sample that comes out are cases that have been around for a long time.

MS. UPTON-DOUGLASS: Just a real quick question. What would be the reason -- I think the numbers are very good, but what would be the reason for not correcting something that was found to be clear error? Would it just be an oversight or would it be a disagreement with the finding?

MR. CONDRO: No. It could be a lot of reasons. Maybe the manager didn't communicate with the examiner. I mean, I don't know. I know that it was a problem when we first started. People didn't like it. I mean, nobody likes to have their cases reviewed once and then they have to come back and have somebody come back and say well, you didn't correct it. So, the job has certain parts of it that are not -- people don't like it, but I don't think there's anything sinister about it.

I think it's just that people in the beginning were just having a tougher time adjusting to it, but like I said, in over the last year or two, they've done a much, much better job of once they say they're going to fix it, it gets fixed.

MR. JENNISON: Chuck, over the years, I've heard of requests for a review of perhaps examiner attorneys and managing attorneys' decisions on certain applications to avoid ex-parte appeals proceedings. To me, it seems like your organization would be a good place to have such a review to avoid going into the ex-parte appeal.

Also, your organization might be a good place for the letter of protest procedure because you're absolute -- you've kind of doing the same thing that the letter of protest is doing, but you're taking it from third parties to say perhaps this is a clear area or mistake, and you seem to be in the trenches doing this thing all of the time.

MR. CONDRO: Well, I don't know about that. All I know is that if we took on any additional role, you'd have to have a lot more staff. Right now, we've got -- it wouldn't even be close because the Department mandates that we look at 4 percent; that's it. If the IG finds out you're looking at less than 4 percent, they're in here and they're giving the office problems.

MR. JENNISON: How many do you have?

MR. CONDRO: Three people.

MR. JENNISON: Counting you.

MR. CONDRO: No. There are three people that review applications. If the office was to take on a significant different role, it would require significant additional staff and I'm not sure even then whether the Department would want an organization that is reviewing work to also be doing -- I mean, I'm not saying this correctly, but reviewing somebody's work should be then be doing it also.

There might be a conflict there. I'm not sure because if you're looking at protests, you're making a judgment. You're sending out something to say -- to an outsider saying yes, we're going to grant your protest; no, we're not going to grant your protest.

In other words, we don't deal with the public and to a certain extent, if we were to handle protest letters, we would be dealing with the public. I'm not saying I'm against it. I'm just saying I don't know how the Department would look at that. We would then all of a sudden be expanding our role from an internal organization which looks at work strictly internally to be starting to tell the public granting protests, not granting protests.

I'm not sure. In fact, if I know the IG like I know them, they would probably object, but I can't say for sure. I don't know.

MR. KRUGMAN: Bob?

MR. SACOFF: Bob Sacoff, for IIPA. Just from an institutional standpoint, I agree with Mr. Condro's reservation. It seems to me that it makes his unit more objective and more somehow credible that it's not actually involved in -- I mean, in the ideal world, yes, they'd find the problems and correct them, but I think from an institutional credibility standpoint, it probably is preferable to separate the two functions.

MR. KRUGMAN: Anybody else?

MR. SIDOTI: I'm not sure I agree. I think with the problems of the letter of protest that I've seen, I wonder if it isn't worth exploring that an independent, non-partisan -- you know, there tends to be, and I'm a great proponent of the positives of the Trademark Office, but sometimes, when there are problems, as you say, it's really tough to correct those problems and there is sometimes on rare occasion the circling of the wagons kind of mentality that occurs, and I wonder if it doesn't beg for more of an independent review of letters of protest of an organization that's just out for quality.

So, again, I don't know if there's a ready solution on it. I think it might be worth exploring though.

MR. CONDRO: Well, the only -- there's probably a middle ground here and that is that the organization could review the decision. I'm not thrilled about that. I'm just saying it could review the decision within the operation.

I think part of the problem is going to be dealing with the public. I know how they feel downtown and I know they're not going to like the internal organization dealing with the public. I don't think they would have any objection if it was within the organization itself.

MR. SIDOTI: I'm not advocating the letter of protest procedure get any more bogged down and complicated. It's just I do wonder whether with the appropriate staffing and the appropriate procedure changes whether it might not be worthwhile to reconsider the whole way it's done. Just a thought.

MR. KRUGMAN: Sharon, are you going to feed us?

(Whereupon, at 12:15 p.m., the meeting in the above-entitled matter was recessed, to reconvene this same date at 1:15 p.m.)

A F T E R N O O N S E S S I O N

(1:25 p.m.)

MR. KRUGMAN: Okay, there's a few folks that are probably still making some calls, but it's 25 after so I think we should sort of plunge into the afternoon session and Bob Anderson is back, as you can see, sitting to my right.

A couple of things from the morning session that I would like to have Bob spend a few minutes discussing is number one, with respect to the proposal by the NTEU representative regarding the examiner's grade level essentially being -- the working grade being raised to 14 on a non-competitive basis. I want Bob to at least give his perspective from the PTO and then after that, it seemed to me that with respect to the quality survey that we looked at and discussed, the obvious question is that they've got this survey now.

They've got these results. What's the office planning on doing about it in terms of implementing some of those recommendations? So, I think Bob would be the appropriate person to give his comments about that.

So, I guess first, let's just if you have any remarks about the union proposal about the non-competitive GS-14.

MR. ANDERSON: I don't know how much background you got on the GS-14. It came about through a study that was done in the Agency by an outfit called NAPA. It's National Association of Public Administrators. They did an objective study covering not only the attorney grade in Trademarks, but several other positions in the PTO.

The outcome of the study regarding attorneys in Trademarks was they indicated that they believed the work being done by our attorneys, or some of the work being done by our attorneys was appropriate to support at grade 14.

Probably, the only point of difference between the Agency and NTEU 245 at this point is how that should be implemented. We have, in fact, brought several new 14s into the organization since the end of the study, but it's been done on a competitive basis. GS-13s apply and they are selected.

We currently have an open announcement that will stay open through December 31 to select additional GS-14s. I understand that Howard mentioned a working grade. I won't into the nuances of federal labor law, but the biggest -- if there is a point of contention between management and the union, it's how 14s would come into the workplace.

One of the things that we believe is that perhaps not every GS-13 is qualified to be a 14. And I think, in essence, that's what we want to deal with. How we deal with it is an open question. As I understood it, Howard indicated that there's been discussions before on topics of this nature and that would probably be a successful way of handling this particular situation.

I think one point of strongest contention between 245 and management goes to the performance appraisal plan. I understand this morning Howard pointed out that even though in the OPM guidelines for GS-14s verses 13s, verses 12s, 11s, and so forth, for attorneys in the federal sector, the biggest difference between position grades is a level of responsibility that an attorney has in an organization.

GS-14s are, generally speaking, held to be very independent and have a very high level of responsibility for their work, whereas a GS-11, which is our starting grade, only does work with a lot of review. As Howard, I think correctly pointed out, the current performance appraisal plan, the differentiation, is largely on the amount of work that an attorney does in the organization.

For several reasons and, in fact, one which will be addressed when we get to the quality part of this thing, I believe, and in general management believes, that the performance appraisal plan for attorneys needs to be changed to properly reflect the level of responsibility that attorneys should have in the organization as their grade increases; i.e., increase the amount of responsibility they have for high quality product, for outcome to their work, so on and so forth. Whereas, a GS-11 attorney would have a lower level of responsibility for that final product currently because it's under review by a senior attorney in the organization.

When I use senior I'm not using -- we have a job title called senior attorney. When I'm using it here I just mean someone with more experience who has attained a different position in the organization.

I think that the grade 14 should be tied to some change in level of responsibility for outcomes -- i.e. work products -- that go through the Patent and Trademark Office. And, in particular, the meeting I went to downtown this morning was to address an INTA concern regarding the quality of work that is being put out by the office.

Now, the study that was done indicates that a large segment of our customer population is very happy with our quality. The ratings were in the 80 percent range for quality of examiner actions. Eighty percent, according to the person doing the study, starts to approach world class levels. On the other hand, there is another 20 percent there that weren't too happy with our work, but I will talk about that a little bit more later.

In any case, I don't think there is any big disagreement between 245 and management about grades for attorneys. The disagreement comes more in how we handle -- how we have the performance appraisal plan and other things reflect the level of responsibility for the higher paid senior attorney in the organization verses an attorney who has just come into the office off the street and is new in the office.

Bottom line is I understand there was no major objection in the meeting to us doing fairly large scale promotions and that will probably happen at some point in the future after we can reconcile some of these other smaller problems.

Any questions on that one?

(No response.)

MR. ANDERSON: Okay, the Westat Quality Survey. I handed out an executive summary trademark quality which was given to Morley -- I believe his name is Weingard -- down at NPR this morning. This covers the basics of what you were briefed out this morning on the Westat study. At the sake of repeating a little bit, there were 14 items that were of major interest to customers of this organization and of the 14 items most strongly related to overall satisfaction, eight related to timeliness.

Received satisfaction ratings of less than 60 percent suggesting that if the organization focuses efforts on timeliness of processing, customer satisfaction would improve. It also points out that problem resolution and timeliness are big problems for us and one of the things I know you have heard this morning was is many customers have problems with individual files at a rate over 50 percent and once a file falls into a problem area we have a hard time extracting that from the problem.

Also that, if we do that well 92 percent of the customers who have problems and have their problems solved with a high level of efficiency are very happy about it and, in fact, it helps improve their view of the office in terms of the quality of our work. On the other hand, most customers didn't fall into that category. That was a relatively small percentage of people.

After I got the results of this, one of the things I've done is I now have someone that reports to me who is taking a look at problem files. If the files hits my desk, we are doing what is called a root cause analysis for that particular file. We are also keeping a database on the results. My goal is to start to pinpoint why files are having problems in the organization.

What she is attempting to do when she does this research is literally track every step of the file through the process, record what happened and see if we can't start identifying the root cause of the problem and then address that problem and get rid of it. I believe that by looking at a relatively small number of files, we can probably solve significant problems across the board and the studies -- now, this has only been going on for about three weeks but what we have seen so far is there is an element of truth to that.

Files that have problems tend to have the same problems which cause them to fall into this kind of never never land where it's hard to get them back out again. Now, the problem that is being addressed there is lost file, lost paper, lost specimens, lost check -- where you have, you yourself or your representative comes to the office, you are trying to find a file record and, lo and behold, we can't find it.

That is not all the problems that everybody has with what we are doing but it apparently is a significant problem for many of our customers. We are going to work on solving that problem. Other things going on, we are improving our automated systems. By the end of November if everything goes right, every incoming application will be scanned as an image, all of the papers in the file, and stored on a server.

If that application gets lost, we will be able to pull down the images of the pages and recreate the file. The other thing it will do is, eventually, these electronic records will be available to Public Search Room users, the certification branch and so forth which means that to do certain things, people will no longer have to pull the paper file wrapper out of examination.

One of the things that this person doing the work for me has already found, files often times get sidetracked because somebody wants to look at a file. The file then gets moved out of the process and doesn't get back into the process efficiently. We hope to solve some of that by making records available electronically, literally anywhere in the agency.

It will also help us now when we lose a file permanently, as some of you may know, we have to call you up and get the record often times. In the future, we shouldn't have to do that so often. Now, unfortunately, this will start a day forward and we already have some 360,000 file wrappers under examination in the organization and, of course, the opportunity for any one of those 360,000 files to get lost is fairly significant.

Other parts of the quality study, even though the examiners got fairly high ratings on quality, we will be addressing quality in the organization. Many of you, I know, have examination problems. What one result of the NPR meeting this morning -- we are going to look at incorporating quality standards into our strategic business plan and in the other parts of the organizational structure, including performance appraisal plans for managers, in particular.

We ill be looking at things like the results from the Office of Trademark Quality Review to see if we can address some of the problems that are pointed up through Chuck Condro review of applications in the office. We will also be working with INTA and I think we have a commitment from them to start addressing what is quality that you want to actually measure.

I mean, one of the problems we always have in the office is there are two sides to each quality issue depending on who you are representing. In particular, in likelihood of confusion, if you are representing the applicant, the applicant is very happy when we don't cite a mark. On the other hand, if you are representing the registrant and we don't cite the mark, you are not very happy with us because you believe it should be cited and everybody knows an ex-parte proceeding, particularly if it doesn't get past the examiner, is much cheaper than filing an inter-parties proceeding at the Trademark Trial and Appeal Board.

What we need to do is to pin down that dichotomy between who is representing whom and get to the underlying issue of what we should be doing and what we shouldn't be doing in ex-parte examination. While I was here this morning I heard a couple of things broached on the topic of letters of protest and at lunch I was in a conversation about the use of Internet resources for examination and one of the things that was broached was the potential of the office in the future perhaps looking at common law refusals on likelihood of confusion because the Internet makes a lot more information available to us.

Some of you may be aware that our examiners are now using the Internet fairly extensively for evidence in applications they are handling. I think as we get more sophisticated in using electronic technology, we have the potential of doing a lot more than we do today, which was one of the issues with the letter of protest.

You are going to see different kinds of things being cited that you would never have seen five years ago simply because we have access to relatively cheap and relatively efficient resources.

Again, I know Howard mentioned the short amount of time we give examining attorneys for examination. On the other hand, as we improve the efficiency of resources, examiners can gain access to much more information when making a likelihood of confusion or a refusal.

Setting a balance between ex-parte examination and what best suits the needs of the registration system I think is a tough issue for us and I think the only way we can get to that is to work with the private sector to get this defined. Hopefully, we have a commitment to do that.

I guess I would be willing to take questions now on the quality study customer survey, in particular. I'm sure there must be some. Yes.

MS. DARE: This isn't so much a question as a comment. One of the things that was very clear that was a problem, and a problem where an implemented solution would create a great increase in customer satisfaction, was the file that has a problem that suddenly seems to be cursed and never kind of gets on the right track again and you try two or three times to make a contact to solve that problem and you just can't do it.

It seems to me there may be kind of three different solutions, but they are all the same type and it's having someone within the agency who is really responsible who, once the file gets de-tracked, puts it back on track and one person becomes the point of contact instead of the number of people who touch a file, in the ordinary process of things. It makes sense to me, simply because of the way I run my practice, that it would be an attorney and that either the individual attorney who was the examining attorney or an attorney who is the designated problem resolving attorney in the law office in question would do that.

Then the third alternative -- because, obviously, you do want your attorneys practicing law as much time as they can and not doing administrative functions -- is that you would designate a problem resolution specialist or ombudsman or goodwill ambassador who could handle these things and at least give meaningful contact to the applicant that the problem was being resolved, that only one phone call had to be made, that a specific set of follow-up actions were recommended.

It seems like we could streamline that process and have a single point of contact and some comfort levels that, once the problem had been identified, it could then be solved instead of -- that it would create and resonate more problems. I think that would help everybody out.

MR. ANDERSON: I think that is probably where we are headed with the person I was just talking about. I am a little hesitant to name them as an ombudsman person yet because I don't have enough structure for it.

The thing I have found a little frustrating -- thus far, I've given her 10 files. The first one was a telephone call to me from an attorney who was extremely frustrated with us. We'd pulled the file from the Official Gazette last November, November of '97, on an examination question. The file, literally, disappeared until just a couple of weeks ago.

I put this person finding out what would go on and, lo and behold, within a period of about three hours the file turned up and was immediately put back into the publication process. And I had her go check, and the file was literally sitting on one person's desk for a period of almost seven months.

I have no idea why that happened, but it suggests to me that we need a process for when a file gets pulled, it will have some special status in our TRAM system or on the cover of the file to make sure that the file gets back where it's supposed to be.

I mean, this somewhat goes to a topic that I think is on the agenda a little later about having centralized file checkout. One of the reasons we are proposing that is to have better control