Conference presentation: Discontinuation and selection: using cure models to evaluate the contribution of selection to sectoral differences in modern method contraceptive discontinuation

This is a blog post based on the presentation I'm giving at PAA this afternoon.

Contraceptive discontinuation is a key indicator of the quality of care provided within a family planning programme. Comparison of headline rates of contraceptive discontinuation can be problematic due to differential prevalences of ‘cured fractions’- women who have a zero probability of discontinuation. These women are heavily concentrated among long term contraceptive method dispensed by public sector providers, and comparison of discontinuation rates without accounting for cured fractions can be misleading. This analysis therefore uses cure models to evaluate contraceptive discontinuation using DHS calendar data and a 7 countries in sub-Saharan Africa. Results indicate that a substantial part of slower rates of discontinuation among long term modern method users can be explained by cured fractions. Conditioning on this, discontinuation rates are higher among users of these methods, potentially indicating a greater rate of side-effect related discontinuation and implications for care provided by public sector providers. 

The fundamental technical piece of understanding we need here is the idea of a 'cure model,' aka a split or two population model. The basic idea is that in standard event history models, we are looking at the rate that a particular event occurs over time. The problem being, that under our two population framework, some of the people we observe are not truly at risk of the event occurring. This is common under circumstances where we are looking at the recurrence of a disease, but we 'cure' a certain proportion of patients (hence the 'cure' model). Under the circumstances we are looking at, contraceptive discontinuation, we can think of women as being 'cured' from contraceptive discontinuation under circumstances where they are

1.  Are practising stopping behaviour (so have no risk of contraceptive abandonment)
2.  Are using a reliable contraceptive diligently (so have no risk of contraceptive failure)
3. Are happy with their current method (so have no risk of method switching) 

This leads us to an analysis where using a definition of contraceptive discontinuation consistent with the DHS that is vulnerable to bias, due to the presence of women fulfilling these criteria. The specific bias we are talking about is illustrated below: we are including women in the risk set who have no risk of experiencing the event, are will continue within our population until the end of time

Why this matters is because the distribution of the cured women is likely to be correlated with certain variables we might be interested in- for instance, there may be variation across country or between different providers. Where we are using continuation rates as an indicator of quality for family planning programmes (Jain 1989, Bruce 1990) this is problematic: some of the 'good' performance is merely reflecting variation in terms of the women served and our comparison is subject to a selection bias.

This is probably the most obvious point of attack for this analysis- particularly for contraceptive switching: it's easy circumstances where a woman changes to a more suitable method. I would, however, whether this really detracts from the idea that there is some degree of programmatic failure involved in that decision. Why was the preferred method not available in the first place? If women change method a lot to a string of 'preferred' methods, is contraceptive counselling really adequate? Besides, even if the change really is desirable, there are still positive risks in the contraceptive change relevant to programme evaluation, such as an elevated risk of accidental pregnancy, regardless of whether the switch is desirable in terms of user satisfaction.

The results of the analysis using the DHS definition of contraceptive discontinuation is presented below. We are making cross-national comparisons here, based on a country selection described in other analyses and using the contraceptive calendar, one of the more reliable means of collecting contraceptive use. Results are presented below, comparing the hazard of contraceptive discontinuation for long term methods (IUD, Male sterilisation, Female sterilisation, Implant) to all other methods for both usual event history and cure models.

Looking at the column for the hazard model, we can see coefficient which are generally below 1 for most countries. This indicates that in most setting, users of of long term contraceptives are less likely to discontinue contraceptive use that women using a short term method (this is a fairly intuitive result). In terms of a naive policy recommendation, this would tend to point us toward encouraging the use of long term methods: where lower rates of contraceptive discontinuation are desirable from a quality perspective this would tend to be a means of lower the rate at which women stop using contraceptive.

However, the column on the right, which accounts for our cured fraction has a slightly more mixed result. Of the 5 countries which had significant coefficients below 1, only 2 remain (Burkina Faso and Malawi). In contrast, we now have 4 coefficients which are above 1, indicating that, accounting for our selection effect, women who use a long term method are more likely to stop using that method than short term method users. Our policy recommendation here is radically different: our call would be to improve the quality of contraceptive counselling for long term method users to reduce their inflating discontinuation rates. This is particularly relevant where there is a greater propensity for women to accept a long term methods, such as in the public sector (Campbell et al 2015). Moreover, it reinforces the conclusions I have made in previous posts

selection effects exist within family planning and that failing to account for these can lead policy makers to overplay their hand when advocating certain solutions. If we want to use genuine evidence based policy we need to think about the nature of behaviours we are generating evidence about: simple regression may not be adequate.

Are imposed institutions exogenous?

This is post is something of a mirror image of previous posts that I have made, where I have talked about the potential endogeneity of demographic behaviours and the institutions within which they occur (see here and here). So the task here is to try and find a circumstance whereby we might be able to identify a set of social institution which were truly exogenous to the populations on which they were imposed- this would allow us to start to get at estimating causal effects of policy on demography. It also means that I get to mention Prussia, the history of which is fascinating for me but possibly boring for everyone else.

Socialism: an exogenous social institution

A natural place to look for these type of institutions would be the European countries of the Soviet bloc, and in particular the Deutsche Demokratische Republik. The major demographic difference here is in terms of non-marital childbearing: non-marital childbearing is much more common the East than the West, a behaviour mirroring the attitudes of the citizens of those regions (Perelli-Harris and Bernardi 2015)

Non marital births within Germany 2009

Source: Klüsener and Goldstein 2012

Legend denotes share of births to unmarried mothers

The argument for why the Socialist institutions we saw in these countries is based on the logic that Socialism was largely imposed on these countries by invading Soviet armies- there was no Miracle of the House of Brandenburg this time (you were warned). As such, the imposition of the Socialist institutions post-war should be exogenous to the underlying characteristics of the country in question. (You cannot make the same claims for the USSR incidentally, the fact that Socialism emerged here following revolution means there are potential endogeneity issues that authors elsewhere (Menaldo 2012) have tried to correct for using instrumental approaches). 

The DDR in particular is of interest since this was the partition of a country which had existed as a single entity pre-war. Englehart et al (2002), for instance, explicitly identify the division between the BRD and the DDR within a pseudo experimental context

The intergenerational transmission of the risk of divorce is a well-known long-term effect of divorce that has been found in many Western societies. Less known is the extent to which different family policies and divorce laws have an effect on the intergenerational transmission of divorce. In this paper, the division of Germany into two separate states from 1949 until 1990, with the consequent development of two very different family policies, is regarded as a natural experiment that enables us to investigate the effect of family policy on the mechanisms underlying the social inheritance of divorce [emphasis mine]. Data from respondents from the former East and West Germany participating in the German Life History Study are analyzed, using multivariate event-history methods. The results indicate that the strength of the intergenerational divorce transmission, when adjusted for differences in the divorce level, was lower in the East than in the West. Differences in marriage age and the timing of first birth, which are partial indicators of family policy, as well as differences in religion, could explain this effect. Furthermore, we found a tendency towards a reduction in the dynamics of divorce transmission over time, both in East and West Germany.

Engelhardt et al 2002. 

A contrasting but related argument is made by Konietzka and Kreyenfeld (2002), who argue that post Wende the replacement of the institutions of the DDR should lead to convergence of East German demographic behaviours with those in the West: the implicit assumption here it seems that there is a causal and exogenous effect of institutions of demographic behaviour 

In contrast to West Germany, where marriage and childbirth have been strongly coupled, the German Democratic Republic (GDR) displayed high rates of non-marital childbearing. Researchers attributed this pattern to “misguided” GDR family policies that encouraged women to remain unmarried after childbirth. With German unification, East Germany’s legal and political institutions — including family policies — were replaced by those of West Germany. Against this background, it was widely expected that east German non-marital birth rates would soon fall to west German levels.

Konietzka D, Kreyenfeld M. 2002. Women’s employment and non-marital

childbearing: a comparison between East and West Germany in the 1990s.

Population 57(2) 331-357

Socialism: an exogenous social institution?

So how good is this research design? According to Klüsener and Goldstein 2012, not very. There are two fundamental problems with the sort of pseudo-experimental design we were driving at earlier.
Firstly, to quote directly from  Klüsener and Goldstein:

The demarcation line between East and West Germany was an outcome of World War II. However, it is important to note that the border between the two German states did not emerge as an incidental freeze-frame of Allied combat positions at the moment of German defeat in the spring of 1945. Indeed, it was a preconceived demarcation line that had been drawn in advance by the Allied victors. The border was based on preexisting administrative entities, such as former federal states and historic territories of principalities (see also Kettenacker, 1989). Thus, while the inner German border became a formal state border in 1949, its roots go much further back in time. Most of the border followed lines that predate the Weimar Republic and the German Empire of 1871, and can be traced back to the emergence of German territorial states in early modern times

Source: Klüsener and Goldstein 2012

As such, the division of the DDR and BRD follows existing divisions which had existed before unification: it is likely that there were some differences between these state in terms of demography, social and cultural, and legal institutions. Subsidiary principles meant that legislation around marriage and cohabitation were decided at a provincial level. Even within the Prussian state there was some degree of heterogeneity between East and West with a contrast between the Preußische Allgemeine Landrecht, and Code Civil in Prussian Rhineland.

At the very best then, our natural experiment has a confounding issue.

The second problem for our research design is that there is some degree of endogeneity between the sort of social institutions post war and demographic behaviour: East German social policy dissociated marriage and childbearing primarily because the conditions for such a change were already in place” (Salles, 2006). In comparison to our first figure, we can observe the nascent differences between the East and West at the formation of the German Empire, long before Socialism and the DDR.

Non marital births within the German Empire, 1878

Source: Klüsener and Goldstein 2012

Legend denotes share of births to unmarried mothers

As such then, the imposition of Socialism and its subsequent collapse within Germany does not represent the natural experiment that we might have hoped for, were we trying to make statements about causal and exogenous effects of policy on demography. We need to look at little harder to find our identification strategy. 

Contraceptive choice and spousal communication

This post is based off a presentation I'm giving at LSHTM later on today, but previews a later presentation at PAA 2017. I've also used preliminary results in interviews, although the presentation version sadly lacks in both throwing tennis balls around and Star Trek references.

The premise we have here is that certain types of fertility awareness contraceptives have been associated with improved couple communication regarding family planning. A lot of this evidence seems to be based on qualitative evidence from India, where male partners reported greater willingness to become involved in family planning, or at the very least an interest in involvement in the regulation of fertility using this method. The exact logic model is not specified here: potentially we are either raising raising awareness regarding family planning or merely flagging opportunities for coitus, but in any case this does seem to present a beneficial side effect: the use of traditional contraception seems to be associated with improved couple communication which has all sorts of benefits.

The problem however, is that it's a reasonably easy to imagine non-causal explanations for this association: couples with better underlying communication are more likely to select methods which require higher levels of communication in the first place. This is particularly true of traditional contraceptives- it is much harder to practice covert contraceptive use using a traditional method and under circumstances where the woman has limited negotiating power almost impossible. We therefore get a classic selection type relationship resulting in an endogenous relationship: both the level of family planning communication and method choice are both outcomes of underlying couple characteristics.

The solution we use here is to take an instrumental variable approach. The instrument we are using here is religion: religious affiliation is associated with method choice (certain religious sects prohibit the use of certain methods) but there is no fundamental reason to believe that spousal communication should be associated religious group. The exact model is represented below: note that the red arrow is the underlying correlation between the error term and the variable of interest, which produces a biased regression estimate, but there is no association between religion and the error (exclusion criterion). 

In terms of operationalisation then: spousal communication is a binary indicator of whether a woman has spoken with her spouse within the past year about family planning, and the 'treatment' here is the contraceptive method type (traditional, modern, none). We make a string of comparisons across countries: these are broadly in line with a DHS analytical paper on spousal communication. Abridged results are in the table below, with coefficients on a probit scale (we need an individual level error to be able to get a correlation matrix) so numbers above zero indicate a higher probability of spousal communication

Controls: Residence, Educational attainment, partner's educational attainment,

 age, partner is older, polygynous marriage, infecund

The results come in three stages: Model I looks at the simple uncontrolled regression, which seems to find consistent improvement in the levels of communication between spouses who use a traditional method compared to non-users (6/7 countries). This relationship is robust to the introduction of a series of controls (Model II: 5/7 countries positive and significant). However, these regression set ups are still not accounting for our endogeneity problem: the beta estimates presented are still biased due to a correlation with the error term in our regression. 

Model III uses an instrument and the significant effect of traditional methods across countries collapses. Compared to Model II, two countries lose significance entirely and two are 'weakened' to show significance at the 5% level only. Benin is the only country where traditional methods improve spousal communication significantly at the 1% level. Interestingly, modern methods a robust to both the introduction of controls and the instrument.

The overall finding then is, I suppose, slightly disappointing: a purported advantage of traditional methods does not in fact exist and is largely explained by selection effects. I'm not sure that there are major policy implications here: spousal communication has never been a major reason for contraceptive method choice, and the purported benefit really was only a serendipitous side effect. For couples where the most suitable method is a traditional one, I'm not sure that this analysis should change that decision. Moreover, there are naturally limitations to this analysis- the most glaring being that the instrument we have is somewhat weak (measures of association are in the order of 0.20). However, it does fundamentally highlight the fact that selection effects exist within family planning and that failing to account for these can lead policy makers to overplay their hand when advocating certain solutions. If we want to use genuine evidence based policy we need to think about the nature of behaviours we are generating evidence about: simple regression may not be adequate.