#19 - Stephan Litschig (Graduate Institute for Policy Studies)
Public Procurement Podcast -
Does an increased audit risk affect corruption practices? iTunesInterview with Stephan Litschig, Associate Professor at the Graduate Institute for Policy Studies, GRIPS, in Tokyo. His research interests are in the area of development, public and political economics. His work has been published in the American Economic Journal: Applied Economics, the Journal of Public Economics and the Journal of Human Resources. He also serves as an Associate Editor of the Journal of Human Resources. Today’s talk is focused on his joint paper with Yves Zamboni called ‘Audit risk and rent extraction, evidence from a randomized evaluation in Brazil’. Transcript I find your paper with Yves Zamboni very interesting and what I would like to know for us to start the conversation with was what led you to carry out this research.The motivation was that we know that let’s say anecdotally, corruption is very prevalent in many countries but it is much less known what the people could do to reduce corruption. And by corruption what I have in mind in particular is in public procurement for example that public officials will take kickbacks in return for steering the contract to certain suppliers but I also have in mind a broader concept of corruption which is rents in the delivery of public services where the fact is that in many countries public servants don’t show up for work, teachers for example or health service providers. So the basic motivation is just that we know that procurement and service delivery in general could be better and then the question was can we change this by increasing the scrutiny that comes from the top basically, increased audit risk. And why doing this in Brazil specifically?So this was an opportunity I had come across the fact that in Brazil they have this peculiar way of auditing local governments which is done at random so there’s literally a lottery that is done that’s actually on television and so that, what that gives you is that it gives you comparable data on audit findings across Brazil, so for a large number of municipalities, and then I basically got in touch with this agency and wrote to them and proposed this and my current co-author was the one who picked it up and said “look, this is interesting, let’s do this together”. So it’s basically the fact that in Brazil they were doing a particular interesting way to conduct these audits and what we did with Yves we designed this study on top of this existing lottery where we added another layer that basically put aside a hundred and twenty municipalities during one year into a high audit risk group so these guys knew that, basically in the control group audit risk per year was something like 5% but if you were in this high audit risk group you had like a 25% chance to be audited over the subsequent year. How was that selection done? Was that selection done by you or specifically was that a purposeful sampling or was it done at random saying “you guys in this year actually have a higher audit risk”?Yeah, exactly. So it was purposefully at random if you want so, you know, they were already doing this random sampling using a literal lottery, like it’s the same machinery that was used for a money lottery in Brazil, that was very popular so they used the same machines, then we also used those machines to set aside a group of municipalities that, because it’s a random sample should have all observable and unobserved characteristics the same as those that were not selected. So then the idea is that if later on once the auditors go in and they find differences in the incidence of irregularities of matters of corruption this has to be because of the only difference between the two groups which is that they were exposed to ex ante higher audit risk. And did they know that in advance, so were they informed?Yes. So that’s an excellent point. So the overall lottery that was regularly done by this audit agency CGU was very well publicized so in that sense we could assume that they knew about it but to make sure that those that were put into this extra scrutiny high audit risk group, that they were aware of that, in addition CGU also sent them a letter and said “by the way, for one year you’ve been selected, you are one out of a hundred and twenty municipalities and at the end of the year we’ll take thirty of you to be audited according to our usual audit procedures”. So they did also receive a letter. In terms of communication of information to those municipalities, so you guys sent a letter but if you’re looking into corruption to the municipality you’re looking into probably I would suspect multiple levels within the organisation, so did you expect the information about the higher audit risk to filter from the top down or just for that information to remain with the person or the persons that received it?So we were agnostic about that, so the idea was that perhaps, so the mayor for sure would receive the letter… Sure, yeah.…but then some of the people in his cabinet would receive it, then, whether he would communicate this to, down to the health offices and all the teachers we were sort of, we didn’t want to micromanage that so we just said “okay, let’s just send this letter”. The key thing that changed was that objectively they were in a high audit risk group so we thought about this but we thought we don’t want to make this too artificial in a sense. Suppose the government would increase the audit intensity, right, they’re not going to send around lots of letters, they’re just going to do it, they might inform some people but we wanted to keep this as real as possible. But it’s true so what that means is that we don’t know how far the information travelled, that is a limitation. In terms of findings, what were your main findings?So we find that when we look at procurement outcomes we find that there is a reduction in both the share of the amount of audited resources that has evidence of corruption, so both in money terms but also when we look at it at the procurement process level and there’s something like 1,600 procurement processes that we have the data on and we find that at that level as well there’s a reduction in the probability that auditors find evidence of corruption. Now what does corruption mean? We basically use, so these auditor reports have been used before in the literature so we use existing corruption codings and one unambiguous example would be the auditors go in and they find that the firm that, you know, received the contract in fact doesn’t exist. So that [Pedro laughs], it’s scary but these things happen, you know. Other… I know.…other things that, right. Yeah, yeah, well, you know, some people are very surprised but I guess you’re more of an expert in this, in this area. But yeah, so other things are when they go in, the auditors go in and they find that the government paid more than what they had agreed to in the auction basically, without gaining any other compensation. I was laughing because a few years ago in Portugal when we started collating the contract data, some people that I know in Portugal they cross-referenced data from that particular database, the contract database with information from the companies registry and they found that certain companies were being awarded contracts even before they existed. That’s why I laughed because I’ve seen that kind of stuff happening before!Excellent. So it’s not only Brazil, yeah.No, I’m pretty sure. You know, so this, you know, I’m getting ahead of myself here but I think in terms of future research there is a lot to be done here, just sort of, just in terms of measuring how frequent does that happen in different countries in different contexts, I think we just know very little about that. And, you know, one reason for that that we know little about it is obviously data, data problems of merging different datasets but the other one is that in most countries the audit agencies would target sectors or firms where they think there’s a high likelihood of corruption and you end up with a very unrepresentative sample of things of what’s actually, what the state is in the overall country. The nice thing about Brazil is that because of this random sampling what we have here is basically a, you can think of it as a random sample so if we find for example that the incidence of corruption is something like let’s say 40% amongst restricted modalities, that’s one of our main findings, we can sort of confidently say that’s more or less what’s going on in the public sector at a local level in Brazil, as opposed to, you know, this reflects the tip of the iceberg or something like that. But, you know, to do that you need the random sampling and that’s, no other country in the world does that so if you want to do comparable studies across, just, you know, compare the incidence of corruption across countries you have to convince public officials to do things in a random way and that’s often very tough. But that’s, in terms of future research that’s sort of where I think this could go. That’s one of the point that I would like to drill down a little bit further which is in terms of processes, which processes have you found out that were more likely to generate corruption or have corruption arise from them?Yeah, so that was an interesting find is that for the first time in our study we have the detailed process level data which means that basically that process is it’s a given purchase, they vary a lot in terms of magnitude but what we know for each purchase is was it done through a modality that restricts competition in some way or not, which is an open auction and you specify in the call what you want and then anybody can pretty much apply. So we group modalities, there’s more than those two but we group them into two and the finding is pretty stark, is that, amongst let’s say public procurement auctions the incidence overall of corruption is much lower, something like 20% and it doesn’t change at all, whereas amongst modalities that restrict competition and give more discretion to the public official, the overall incidence in the control will be something like 40% and there we find the reduction, it only occurs there, something like 15 percentage points. So it’s only the modalities that restrict competition, that afford more discretion that, where this corruption seems to be going on and where it is affected by higher audit risk. Does Brazil have financial thresholds in terms of determining which kind of modalities you’re going to use, if you’re going to use the auctions or you’re going to use the restrictive competition ones?Yes. Or is it just done by case-by-case scenario?No. There are thresholds and one of the things that the auditors look at and that we also, that is part of some of our corruption measures is in fact whether they basically fraction the purchase to avoid those thresholds. So there are thresholds. Because that is fascinating, especially, or even in Europe because the financial thresholds in Europe are quite high so you only apply rules for contracts above 100 and something, for goods and services 100 and something thousand Euros if I’m not wrong or more, or even more if you’re a local authority. And I’ve always thought that it was not a very good idea, especially on corruption grounds but I’ve been writing about that area over the last few years and I’m going to revisit it again and you’ve just given me a new piece of the puzzle for me to work with which is actually where there is more discretion for public procurers we are creating the conditions to generate more corruption or at least make corruption more likely?So that’s what our findings suggest for the first time that at least in Brazil in this particular context this discretion tends to be abused, okay. Now how externally valid is that? As you say I think that’s… Yeah, we don’t know?That’s research, we don’t know, that’s right. Okay. But in that sense I’m writing about these ideas and obviously this area of public procurement from a legal perspective so I can afford the limited availability of data to try and shape my thoughts, although obviously I want it to be sustained by evidence and…?Mm, and of course, of course. Right, okay. So I’ve got a few extra comments for you.Sorry Pedro, can I, because you asked and I’ve been going off topic I think so the paper is about procurement but if you don’t mind? Yes, sure.One of the things we also look at is this idea that when you increase the audit risk service providers are more likely to actually show up for work, okay. So there’s, it’s not corruption in terms of stealing but corruption in the sense of you’re paid to do a job and you don’t do it so, and the reason we can do that is because the CGU audits they include that as a standard procedure for them to basically also try to assess whether for example in health service delivery the officials, and these are going to be doctors and nurses but also going to be health workers that are supposed to visit the health facility users preventively, so the idea is that it’s cheaper to prevent than to cure but to, the prevention requires that these guys go out there and visit the families to find out whether someone is ill so that the prevention can actually take place and that’s costly of course for these agents. So what we looked at there or what the auditors looked at is they ask a random sample of potential service users in a given area they make household visits and they ask “are you receiving these visits from these preventative healthcare workers?” They also ask “when you go to the health post is it open when it should be open and when you go there is the doctor there?” These kind of questions so, and they ask some questions in control and treatment municipalities, okay. Now what we find on that front is that although in the control group there is not perfect performance in the sense that people do report that the health post is not always open as it should, the doctor is not always there, the increase in audit risk has no effect on that performance, on that compliance with your stipulated job requirements. In stark contrast to what we find in procurement, we find this reduction.So what we, we were puzzled by this, it was we didn’t expect that but then what we do in the paper is we obtained a standard economic model and we see whether we can rationalise these findings. And one explanation for these results could be that when you get audited in procurement basically you can’t argue with those findings, you know. If the auditor finds that the firm that you awarded the contract to doesn’t exist there’s not much more you can say, whereas if the auditor goes in and he finds that people report that the doctors were not in the health centre when they should have been, the doctors are going to say “no, I was there” and the audit finding doesn’t generate that kind of hard evidence that would make people work. So we think that ex post we can rationalise our findings so the fact that health service providers might just not be worried by the audit because they can always say “well no, no, I was there” or “I was in another room” or “I was, I don’t know, I was visiting a family out in the field” so it just doesn’t generate the same kind of evidence. And so that, you know, what that, it just, I guess it also, it’s not, obviously it’s not clear how general the finding is but it raises this issue that the quality of audits can be useful to monitor certain things but not others. And then that has implications for audit design so perhaps the auditors can save these visits because at the end what’s being generated is not really useful that much.Anyway, so I just wanted to mention that. I know this is a procurement audience but because the increased audit risk happened for everybody in the entire municipality and because the auditors look at these things as a matter of routine we thought that it’s useful to look at that as well, even though it’s a null result we think it’s interesting. That is interesting and I was wondering as I was listening to you if it may be that the consequences are actually as far as I know very different in case you are found involved in corruption, actual corruption in a public procurement process or just not doing your work properly?You’re exactly right. So there’s an additional reason which is that if you’re found stealing from any public procurement you can basically go to jail, whereas even if you’re convicted that you were shirking on the job you might lose your job at most. At most, yeah. You know, the sanctions are just less as well. So you’re exactly right, that will be another explanation. To be fair there’s also the explanation that as you said earlier, right, information might not have travelled to those guys, could be that the… They did not know?They didn’t know they were not aware of this audit thing and for them they were not treated in some sense. And, you know, we won’t know [laughter] so… I’ve got another question here, a little bit more specific is that I think it’s on page 11 or 12 that you say that corruption or at least stealing less today does not imply that more can be stolen tomorrow, so you argue that because the federal funds cannot be saved for periods longer than the budget year that there’s no possibility that someone will not steal today so that they can steal more the next year because they’re just going to lose the budget?Uh huh. I agree with you, I think that is true but I was wondering if you found something within, any difference within the same budget year? Let me explain, what I’ve seen working with public procurement officials in different countries is that sometimes when we get close to the end of the budget year you still have some of your budget left and you need to spend it, you need to spend it and you don’t usually spend it in the best way possible, so that would increase the likelihood that someone would be tempted to take some money or be corrupted in one way or another. So what I’m wondering is if within the same budget year or fiscal year have you seen any differences or potential differences in terms of one is more, when is corruption more likely to occur?Mm. It’s an excellent question but without data… Without data, yes.We are bit limited on that because we don’t know basically when the purchase was decided, you would need monthly data in some sense, right? You need to know when was, and that we just don’t have. I think for other countries, I think for Chile there’s actually a study that basically shows what you’re saying. Maybe you had that study in mind? But so … No I don’t, I don’t. I would like to know more about it so if you know it just give me the information afterwards and I’ll look it up?Yes. I don’t have the authors off the top of my head right now but this has been empirically documented. Now I think what they look at if I remember correctly is just, do they, so I’m not sure they actually have audit findings on these transactions, I’m not sure if they can say that it’s more corrupt if it’s spent towards the end of the year but things do happen that are special at the end of the year, that much I remember. Okay, very well. One final question, next steps and future research plans in this area, do you have any? Do you think that you’re going to still revise the paper one more time or…?Well the truth is that it’s not published yet so I’m sure I’ll revise it, that’s just, but that’s nature of the game and, you know, in fairness to the referees it usually gets better. But the paper is what it is. What I do find though is that academically people tend to, at least the referees I’ve got, they often say “well it’s not really surprising, we already know that an increase in audit risk will reduce corruption, it’s sort of obvious” and I think that’s just wrong and I think it’s actually, it hurts public policy because you just assume these things and you don’t study it. The fact is that we don’t, we know very little about the incidence of corruption in procurement and service delivery more generally across the world and then what works to reduce it, if it has worked in one context is it going to work in another one? We just have no idea. So I, there’s the, the short answer to this is that I think I’ll continue working on this topic trying to see whether more monitoring or other kind of monitoring or even also incentive schemes can improve service delivery and reduce corruption but I’ll have to find willing partners and that’s, you know, that is the, you have to work, to do this kind of stuff you need connections with governments and that’s, that is sort of the major constraint, okay. So I can say I want to do that, the question is going to be am I going to find governments that are willing to do that kind of stuff and I can just, all I can do is try. So that’s one.Another area where this can go where it’s a little bit, a different question is the fact that public procurement has in recent years moved very much online so e-procurement where from a monitoring perspective what’s interesting about that is that there is much more information now about let’s say the awarding stage, right, so you can basically go online and see what kind of contracts are available for which you can bid, you can see afterwards, you know, who won, so there’s just much more transparency in the entire awarding stage and I have a project with two other co-authors where we try to see when, under this increased transparency in the awarding stage corruption or irregularities get shifted more towards the execution stage because in order to know whether the price agreed was actually the price paid or whether the quality was according to what was agreed or not, whether it was substandard quality in the execution, you actually, you still need an audit. This whole movement of e-procurement hasn’t changed the fact that we know very little about what’s going on in the execution stage and there’s our worry that just the focus of corruption might shift towards execution. So that’s just, you know, putting sort of a question out there that we’re working on but we haven’t, I don’t have any results but I think that’s, in the future that is interesting, So less or increased transparency at the awarding stage might actually shift, just shift things around and overall towards the execution and that, and overall you might not get much of a benefit. So that’s a worry that we have and we want to investigate that. That’s a very good couple of points that you’ve made there for us to conclude the show on. I would just like to say that I agree with you that the more transparency that we have during the awarding stage, the more likely it is that whatever, let’s call it irregular behaviour or illegal behaviour taken by agents is going to move elsewhere. You are right in looking at the execution stage because execution stage unfortunately so far has not been subject to as much scrutiny as the award stage and that for me is a mistake. It’s a big, big mistake in terms of policy and I’ve been against that view of procurement, that procurement is only the award stage and not execution for many years. On the other hand something that you may not be aware of is that at least in Europe under the new EU Directives it’s now possible in variable terms unfortunately in my view for contracting authorities to have conversation with suppliers even before they open the award stage, so that means that there’s another, there’s another moment where corruption can occur which is actually before the award procedure. So even if you have let’s say an open procedure, so what you guys would call an open auction for example, if you have an open procedure, very transparent, very clear, you look at it, just at the procedure and you will not find any corruption but you don’t know that for example one of the bidders was told in advance what was going to be included there and for example shaped the actual award criteria.Right, so the buyer can tailor the call to whatever they, and you’re saying this is legal in Europe? What I’m saying is that it’s legal nowadays for the contracting authority to have conversations with suppliers before it launches a tender and the way that it is drafted in the directive it’s very, very liberal and way too much liberal for my taste.Interesting. But so is this a change that’s happening? Yes.Or are you, ah. Yes, it is a change. I mean economic operators know but contracting authorities have been asking for this for many years, they say that if they are forced to use an open procedure, say an open auction, that is very difficult for them to know in advance what they really need and how to spec it out correctly so they need to talk with the market in advance, so if they talk with the market in advance the procurement is going to be better and I just say bollocks, you’re just creating the conditions for undue influence to occur. But that’s another area that is coming up at least as a potential area of research in corruption in public procurement.Well very interesting. Thanks a lot for letting me know. Well, you know, let’s keep in touch. I mean clearly I have a lot to learn about what’s going on in Europe. Okay. Thank you very much Stephan, was a pleasure talking with you.Thanks Pedro. See you.