Abstract: Multilevel modelling techniques such as random models or fixed effect are increasingly used in social sciences and demography to both account for clustering within higher level aggregations and evaluate the interaction between individual and contextual information. While this is justifiable in some studies, the extension of multilevel models to national level analysis — and particularly cross-national comparative analysis — is problematic and can hamper the understanding of the interplay between individual and country level characteristics. This paper proposes an alternative approach, which allocates countries to classes based on economic, labour market and policy characteristics. Classes influence the profiles of three key demographic behaviours at a sub-national level: marriage, cohabitation and first birth timing. Woman level data are drawn from a subset of the Harmonized Histories dataset, and national level information from the GGP contextual database. In this example, three country classes are extracted reflecting two Western patterns and an Eastern pattern, divided approximately along the Hajnal line. While Western countries tend to exhibit higher levels of family allowances albeit accounting for a lower share of spending which is associated with lower marriage and later fertility, Eastern countries generally show a higher share of spending but at lower absolute levels with lower cohabitation rates and early fertility
Link to the full paper can be found here. This is open access, but I've included some thoughts below as well.
This paper is somewhat related to previous grumbles about the way in which demographic behaviour and institutions are modelled, or at least talked about. For full disclosure, this paper does not deal with the sort of endogeneity problems that I have talked about before: it does however I think provide a more coherant framework for analysing the interaction between national level policy and individual level demographic behaviour.
One of the major innovations in statistics has been the use of multilevel models. Random effects type models will typically deal with some degree of dependence of individual level outcomes on the context within which they are located. Examples of this can be found in schooling: there is variation in pupil level outcomes which is not explained by standard controls such as receipt of Free School Meals (poverty measure), SEN, ethnicity, etc.. Multilevel models are able to partition into individual (attributable to the pupil) and school level variation (This, as I understand it, is the basis of the Contextual Value Added measure in UK school league tables: the CVA is the school level residual in a random effects multilevel model). This is incorporated as an additional error term (often with a Normal distribution for convenience). While this works well for schools (there are enough of them that the school level errors can approximate a Normal distribution), whether this applies to countries is more questionable. It takes a lot of work to get even a dozen or so countries in a form where they can be modelled together (for instance see the excellent Harmonized Histories); but we are still below the sort of numbers where we can reasonably expect the CLT to apply. More problematic is the consistency between the model assumptions and the application: purposive selection of countries for analysis seems strongly at odds that our observations follow the sort of stochastic assumptions required for the model, the endogeneity problems seem to contradict iid assumptions of the residuals. In sum, the application of random effects models seems like too serious a violation of Gauss-Markov assumptions to be tolerable: obviously all models are wrong but there needs to be some sort of consistency between our theory and estimated models. We're not Milton Friedman. The other problem is that since we often want to make statements about cross national variation, the fact that this variation of interest is now subsumed into an error term tends to mean that actually interpreting variation between countries is tough
Fixed effects models are slightly better in that they require fewer assumptions of that nature, but still don't get all of the to addressing the sort of problems we might have. Usual concerns about statistical efficiency are probably not relevant here: the small number of countries probably means that there are not many efficiency gains to be made including a variance term over 7 dummies. However, is the fact that if we want to make statements about the effect of national level policy, the policy indicator is now fully confounded by the country fixed effect, so we have an identification problem. Unless we are including the fixed effect just to remove noise (so why are we taking a multilevel approach anyway?) this is a major research drawback.
The proposed solution is something of a hybrid: I use national level policy indicators to form latent classes (shown below) and then model demographic behaviours dependent on the country level class.
Lyons-Amos (2016)
There are a number of advantages to this approach: we aren't purely tied to the reasonably strict assumption of the random effects model: the latent classes hare discrete to we can account for clumpiness in the 'residual.' Also, these classes are qualitatively interpretable: the use of policy indicators means that we can see what the class actually means.
Table 3: Estimated mean levels of country level indicators by latent class
Lyons-Amos (2016)
Naturally there are some limitations in what we are able to do with these models, which I talk about in the paper: this is means to be a step on the way, not a complete solution. Whether this application is relevant as well remains to be seen- the added difficulty in estimating the latent class model will be a turn off for many. However, models need to be as simple as possible, but no simpler. I'm not sure whether for many demographers simply running MLwiN or xtmixed adequately addressed the latter concern.
