Do Teachers Respond to Incentives? An Experiment in Sachkhere Reveals All

By Zurab Abramishvili

The ISET Economist, a blog about economics in Georgia and the South Caucasus by the International School of Economics at TSU (ISET)

What can bring the brightest among Georgian university graduates to the country’s public schools? While money alone may not do the trick, it is difficult to see a solution that does not represent a radical departure from the current remuneration system which places teachers – who hold the keys to Georgia’s future as a nation – at the very bottom of the social ladder. Not only do teachers remain the lowest paid category of Georgian workers but the gap between the annual average wage in education and other sectors of the economy has been widening over time (see chart).

Of course, money matters not only for the career choices of young university graduates. It also affects the incentives of those currently employed in the system, bringing them to appreciate (or not) their jobs, and make greater (or lesser) effort to master new skills and prepare for classes. And, if sprinkled indiscriminately at the system, money can also cause the worst teachers to delay retirement and put up a political fight for jobs they might hate and for which they have no qualifications.

What we offer in this article is a glimpse into the impact of money on teachers’ incentives and performance, which can be obtained through a careful evaluation of a policy experiment conducted over several years in Sachkhere, a tiny municipality (38,000 people according to 2014 census) in northern Imereti.

THE EXPERIMENT DESCRIBED

Sachkhere is the birthplace of Bidzina Ivanishvili, a well-known Georgian entrepreneur, politician and philanthropist who made his native village Chorvila a center of his charitable activities in Sachkhere and Georgia. Ivanishvili is said to have been paying his former neighbors’ gas and electricity bills and ensuring that their homes have fridges and washing machines. More significantly for our purposes, in 2007-2011, Ivanishvili’s charitable foundation singled out Sachkhere’s public schools as a subject of its philanthropic intervention, topping up the salaries of more than 800 school personnel in all of Sachkhere’s 32 schools.

Considering the dismal level of official salaries, the amounts provided were not trivial: 500 GEL/month for school principals, 400 for vice-principals, 300 for rank-and-file teachers, and 150 GEL for other (technical) staff. Using available data on the number of school personnel in the municipality, we estimate that over five years Ivanishvili’s investment totaled about 19 million GEL.

The fact that the project targeted only one municipality (Sachkhere) and had no direct effect on any of the bordering districts, allows us to conduct a rigorous impact assessment of Ivanishvili’s intervention. We do so by comparing the educational achievements of school graduates in two very similar neighboring municipalities, one of which (Sachkhere) received the “money treatment” and the other (Chiatura) did not. As a proxy of educational achievement we used data on university enrollment rates – the ratio of school graduates enrolled in Georgian universities. 

DOES MONEY REALLY MATTER?

A visual inspection of the data (see chart) suggests that, as far as university enrollment rates are concerned, Sachkhere’s lagged behind Chiatura until 2007; it catches up by 2009 and overtakes it in 2010. (We ignore the 2008 data since this was an odd year in which almost all Georgian schools shifted to a 12-year format and most students did not sit university entrance exams).

A proper difference-in-differences estimation shows that, over 4 years, Ivanishvili’s intervention increased Sachkhere district’s schooling performance by about 10%. Importantly, this result is statistically significant at the 95% confidence level, suggesting that money does affect schools’ performance, and does so very quickly.

One can, of course, question the validity of our results by claiming that other factors contributed to the relative improvement in Sachkhere’s schooling performance. For example, Sachkhere was the subject of many other interventions by Ivanishvili’s foundation, creating jobs and improving people’s livelihoods. It is quite possible that these interventions triggered a process of internal migration with many motivated people – skilled workers and entrepreneurs – moving or returning to Sachkhere from other parts of Georgia. And if a stronger population moved into Sachkhere as a result of Ivanishvili’s activity in the district, this could have positively affected university enrolments and other socio-economic outcomes (particularly so if such “brain gain” was happening at the expense of Chiatura and other neighboring regions).

Unfortunately, these interesting hypotheses cannot be tested given the absence of goof quality internal migration data. In any case, given that Ivanishvili’s general philanthropic activities in the region date back to early 2000s while improvements in university enrollment did not kick in until 2009, it does seem plausible that money spent on educators had had a tangible impact on school performance.

HOW WELL WAS IVANISHVILI’S MONEY SPENT?

19 mln GEL is not a small amount of money considering the size of Sachkhere. While effective, Ivanishvili’s intervention – a near doubling of remuneration regardless of personal characteristics or performance – may be not the most efficient way to achieve improved learning outcomes and university enrollment. The question of efficiency is particularly important if we want to go beyond piloting to a full-blown national reform (according to our calculations, the cost of fully scaling up Sachkhere’s experiment amounts to about 370 million GEL per year, much more than the Georgian budget can carry at present).

The first thing to note (see table) is that Sachkhere seems to have experienced a disproportionate increase in the number of technical personnel. Some of this increase may reflect opportunistic behavior and nepotism. Given the lack of monitoring and the fact that disbursements were not conditioned on the quality of performance, Ivanishvili’s intervention may have created perverse incentives for school personnel: to keep their jobs no matter what, and share the benefits with friends and relatives.

Table: The number of school personnel in Sachkhere and Chiatura, average over 2007-2011

Source: Ministry of Education and Science of Georgia (* refers to approximations)

A straightforward conclusion from these numbers and the informal interviews we conducted with Sachkhere’s school teachers, is that money should be better targeted:

• At schools that exhibit improved performance (to be rigorously monitored);
• At particular teaching professions (sciences, etc.) that may be in short supply in order to create incentives for new teachers to enter the market;
• At the best teachers in each profession to make sure that performance in the classroom and investment in own skills is properly rewarded (and lack of effort is punished!).

It goes without saying that better targeting requires investment in administrative costs (to implement a rigorous monitoring and evaluation system) beyond what was envisaged by Mr. Ivanishvili’s experiment. The challenge, however, is to keep such expenses to the minimum.

Finally, to affect the career choices of young Georgians, no reform should be conceived as a short-lived experiment. Once adjusted, teachers’ compensation should be maintained at the newly established level through an appropriate indexation mechanism. This is the only way for the Georgian teaching profession to regain (and keep) its social recognition and status.

Zurab (Zura) Abramishvili was born in Tbilisi but both his parents grew up in Sachkhere’s Merjevi village. In addition to statistical data, the study is based on informal interviews with school teachers conducted by Zura in Merjevi and Sairkhe villages. A senior researcher with ISET Policy Institute’s Social Policy Research Center, Zura graduated from the prestigious Komarovi Math School in Tbilisi. He holds an MA degree in Economics from ISET and is about to defend his PhD dissertation at the Center for Economic Research and Graduate Education in Prague.