McKerchar, Margaret --- "Philosophical Paradigms, Inquiry Strategies and Knowledge Claims: Applying the Principles of Research Design and Conduct to Taxation" [2009] UNSWLRS 31

Last Updated: 30 August 2009

Philosophical Paradigms, Inquiry Strategies and Knowledge Claims: Applying the Principles of Research Design and Conduct to Taxation

Margaret McKerchar[∗]

Citation

This paper was published in the eJournal of Tax Research (2008) vol. 6, no. 1, pp. 5-22.

Abstract

Taxation is a fundamental part of everyday life and it comes as no surprise that it attracts great interest from policymakers, academics, business and the wider community both in Australia and overseas. However, those interested in tax research come from very diverse discipline backgrounds including law, accounting, economics, political science, psychology and philosophy. The prior learning of many tax researchers does tend to be in the study and application of the law and typically they have little training in or exposure to the detail of the theory and practice of research design. This is a limitation for both academics and for the growing body of research students that they are being called upon to supervise. There is capacity to improve the capability of tax researchers by evaluating best practice in closely related disciplines and exploring that of other disciplines that could have relevance to taxation.

The paper explores the fundamental aspects of research design, including a range of philosophical paradigms and strategies of inquiries that could have application to taxation. The framing of research questions (or hypotheses) is considered, as is the need for alignment between research objectives, research questions, philosophical paradigms, strategies of inquiry and knowledge claims. The application of mixed-method designs is also considered. Pitfalls in research design to be avoided are discussed as are suggestions by which the robustness of tax research can be improved.

1. Introduction

Taxation, with its intrigues and intricacies, is an important part of everyday life. Its shape and form is a reflection of the shared values, goals and aspirations of society, and a means by which its members are bound together. When seeking to understand almost any aspect of taxation, it must be borne in mind that it is much more than the study of the revenue law itself. Taxation is not a discipline in its own right, but a social phenomenon that can be studied through various disciplinary lenses. Commonly, taxation attracts researchers from the disciplines of law, accounting, economics, political science, psychology and philosophy. These disciplinary backgrounds are each understandably narrow and, in spite of researchers being no doubt experts in their respective fields, it can be challenging to apply their skills and knowledge to the complexities of the research problems that emanate from the study of taxation. For example, legal researchers may be well qualified to study the meaning of the letter of the law, but find that they are not quite so well equipped to study how people respond to the law. This in turn may well impinge on any recommendations that researchers may make for law reform. Similarly, an accounting researcher may be experienced in reading data in quantitative form, but not understand the principles of survey design or statistical analysis. This in turn could lead to knowledge claims or conclusions being made or drawn that are not valid.

That is, taxation is complex and researchers in this field are often not fully equipped to grapple with its multidimensional nature, particularly when it involves the study of human behaviour. There are various philosophical research paradigms and strategies of inquiry, the theory and fundamental principles of which a good researcher in taxation (from whatever discipline) needs to understand in order to be able to apply them with confidence. This in turn should allow the researcher to make informed decisions as to their appropriateness in given contexts, and to apply them with rigour. Every researcher is trying to make a meaningful contribution to the body of knowledge and this is best done by producing research of the highest quality. This requires attention to detail when it comes to the design, conduct and writing up of taxation research. The purpose of the paper is to explore these fundamental principles of good research design and conduct in the context of taxation and provide practical examples of best practice. It is hoped that the paper does stimulate further consideration of current practices and also provides some practical guidance for tax researchers.

Following on from the introduction in part 1 of this paper, the philosophical paradigms of research that are considered most relevant to taxation and the fundamental aspects of research design are explored in part 2. The strategies of inquiry used in quantitative research and the knowledge claims that are consistent with this paradigm are considered in part 3. Qualitative research and legal research are discussed in parts 4 and 5 respectively. Other less traditional research approaches including the role of mixed method research are examined in part 6. Part 7 includes a summary of the key concepts covered in the paper and some questions that may help keep tax researchers on track and on the pathway to discovery.

2. Philosophical Paradigms and Research Design

There is a substantial body of literature on the design and conduct of research from which it can be said that there are traditionally two core research philosophical paradigms. These paradigms, or organising frameworks[1] or disciplinary matrices[2] that guide researchers, are commonly referred to as positivism and interpretivism. A research paradigm has its own identifying characteristics, its own ways and means (methods and practices) that create expectations about the nature and conduct of research which it embraces. Grix maintains that all research takes place within a paradigm, whether it is explicitly stated or not.[3] Paradigm choice is by and large a reflection of how the researcher views the world (ontology) and believes that knowledge is created (epistemology). That is, these implicit beliefs, along with the researcher’s disciplinary focus and past experiences, will influence his or her philosophical approach to research, even before the topic is chosen.[4] Whilst every researcher will undoubtedly have a paradigm preference, it is important to keep an open mind to the work of researchers in both tax and other disciplines, particularly where their endeavours can potentially contribute. Further, an effective researcher should be flexible enough to be able to work within the most appropriate paradigm given the nature of the research problem under investigation.

In the context of the social sciences, positivism is adopted by those researchers who seek objectivity in their explanation of social reality. In this paradigm the researcher is viewed as detached from the subjects under study and the explanations derived are based on empirical evidence and tested theories. That is, the knowledge produced by a positivist approach is based on deductive reasoning whereby the researcher follows a precise and structured process leading to the identification of causal relationships, logical conclusions and the making of predictions according to various confidence levels. The positivist approach has been described as being based on a realist, foundationalist ontology which views the world as existing independently of our knowledge of it.[5]

In contrast, interpretivism is based on the assumption that the researcher cannot be detached from the subjects being studied. It is sometimes referred to as anti-positivism. As a research paradigm, interpretivism provides an understanding of social reality that is based on the subjective interpretation of the researcher. It does not provide a hard and fast explanation from which causal relationships can be identified and predictions made. Denscombe[6] describes the explanations of an interpretivist researcher as likely to be messy and open-ended rather than being nice, neat and complete.

It follows that the researcher whose philosophical paradigm is best described as positivist is more likely to adopt a quantitative methodology[7], whilst the interpretivist researcher could be expected to employ a qualitative methodology. The parallels are readily apparent. Quantitative research is empirical in nature, relies on deductive reasoning, is commonly used in the sciences, and has been practised as far back as Hippocrates c.450BC.[8] In contrast, qualitative research began in the 1900s, is more commonly used in the social sciences and is a complex and still evolving paradigm.[9] Qualitative research requires inductive reasoning to be employed rather than logic, and often calls for more creative and indirect means of collecting data or evidence.[10]

However, whilst these two paradigms are quite philosophically opposed, it has been recognised since the 1970s that a continuum exists between them in which other paradigms can and do exist. For example, critical realism and pragmatism both represent philosophical approaches that lie in the middle ground. A critical realist seeks to answer both the ‘how’ and ‘why’ questions. In terms of the underlying ontology, a critical realist sees greater complexity in the relationships under study, going beyond the depths of empirical realism. Researchers who subscribe to this paradigm would typically allow the research design to be driven by what was wanted to be learnt, rather than to be pre-ordained.[11] A pragmatist has a similar approach and freely chooses the methods, techniques and procedures that best meet the needs and purposes of the research.[12] Typically, the paradigms of critical realism and pragmatism are apparent in mixed method research where the researcher draws from both extremes of the continuum.

Axinn and Pearce[13] argue that the dichotomous unidimensional distinction between quantitative and qualitative research is unhelpful as it is ultimately only a distinction between whether or not the data is coded into numbers or into text. However, this argument does seem to be too simplistic as it is clear that different methodologies produce different research designs and follow different conventions that are expected by both our peers and our wider audiences. Sarantakos[14] explains that it is the underlying theoretical structure that characterises the research and that this is shaped by the different ontological and epistemological prescriptions. That is, methodology is much more than, and cannot be determined by, the nature of data presentation.

No doubt by now readers from a law background are asking – ‘but where do we fit?’ It is true, the body of literature on research design is focused heavily on the sciences and the social sciences and there is considerably less said that appears directly relevant to the discipline of law. In terms of paradigms, it is reasonable to posit that legal research could be anywhere on this continuum. It could be positivist and employ a quantitative methodology based on empirical evidence – for example, how often has the law changed this century, how often has a particular section been the subject of a legal dispute, how long are various sections, or how easy is a piece of legislation to read? In contrast, it could be interpretivist and based on social construct – for example, what impact has the introduction of a baby bonus had on fertility rates in Australia, or has the farm management bond scheme helped primary producers better manage their financial self-reliance? Critical realism or pragmatism may well offer a more comfortable paradigm fit for legal researchers than either of the two extremes of positivism and interpretivism. An alternate view is to posit legal research as a different paradigm altogether (for example, socio-legal research[15]and possibly one that lies outside of the standard continuum that has been discussed thus far. This notion of best fit and alternative paradigms in the context of legal research will be considered further in part 5.

Putting aside these philosophical differences, it is readily apparent from the literature that different disciplines do fit more naturally under different paradigms and that disciplines themselves do differ in the emphasis they place on the role and position of theory in research.[16] Whilst challenging the discipline-based principles can bring about change - after all, the rules are not set in concrete - it can be a perilous pathway if the researcher has not based the chosen research methodology on a solid and defensible foundation.

Irrespective of whether or not the researcher is embarking on a ground-breaking methodological pathway or adopting a more traditional paradigm, the research must have a clear purpose at the outset. This may come from the review of the literature in which a gap of knowledge, to be addressed by the research, has been identified. This is a top-down, or funnel, approach whereby from the researcher starts by reading broadly in the area, then drills down to a very specific and narrow aspect that is to be the objective of the research. Alternatively, if there is a broader area of interest, for example, the taxation laws relating to trusts in Australia, it is still worthwhile and infinitely sensible to check the literature early in the piece to identify who else is researching in the area, whether or not there remains scope to make a valid contribution to the field (for example, by adopting a different approach to the topic), or if there is a specific gap of knowledge apparent that offers potential for further research. Further, having a purpose does help the researcher know when the point of completion has been reached and to measure (and rejoice in) progress along the way.

Once identified, the purpose of the research can be expressed as an overarching problem or research question (for example, what are the impacts of employment-related taxes in Australia?). It can then be broken down into smaller defensible issues or questions (for example, consideration could be given to taxes imposed at federal and/or state level; to impacts from different perspectives such as the community, employees, small business employers and/or large business employers; to industry sectors; to impact according to each tax measure (or limited to just one), to short-term and long-term impacts; and/or to alternative models) can help shape the design and give the researcher a clear roadmap. This process of refining the topic should help the researcher place boundaries around the research – for example, the decision may be made that the gap of knowledge exists in respect of state tax laws, and the purpose narrowed accordingly. Going forward, this can help the researcher stay on track and get to the end of the research without unnecessary detours and delays. It goes without saying that these smaller questions must all contribute to resolving the overarching problem. If not, but they are still of interest to the researcher, consider putting them aside for the time being.

With a clear vision of the purpose of the research and its dimensions, the process of designing the research is fairly straightforward. Again, the design of the research should be in such a way that:

• its methodology fits within a paradigm that is (ideally) understood and accepted by others;
• it has a fundamental framework or structure to its conduct;
• it employs appropriate strategies of inquiry or research methods;
• it allows for knowledge claims to be made that are both consistent with the strategy of inquiry; and
• it allows the researcher to answer the research question(s) and meet his or her objectives.

In both traditional and non-traditional research designs the researcher needs to be critically reviewing the chosen methodology looking for its flaws and weaknesses and seeking to address them until personally satisfied and confident in the appropriateness of the chosen methodology.

Apart from issues of appropriateness, it is fundamental that there is clear alignment between research problem and research design if the research is to have theoretical rigour.[17] It is advisable not to lock in prematurely to a particular design, particularly without considering the alternatives. To achieve the best alignment, it may mean that the researcher needs to develop new skills or else revise the purpose of the research and/or the research questions at the outset. Are they achievable? Time spent at the outset revising and refining all the design details is well worth the investment and can save a lot of headaches down the track. One important design detail is (are) the strategy(ies) of inquiry to be employed and there exists a vast array of possibilities. The next four parts of the paper explore a range of them in more detail according to their underlying philosophical paradigm or methodology – be it quantitative, qualitative, legal research or mixed method.

3. Quantitative Methodology

Research designs that embrace quantitative methodology typically use various forms of experiments and surveys as their main strategies of inquiry. These strategies are generally appropriate where the purpose of the research is to relate or compare variables. The purpose generally is to identify if a cause and effect relationship exists, and then to make generalisations about the relationship in the context of a broader population. For example, is the level of taxpayer compliance related to the tax rate? Typically, null hypotheses are developed for each relationship being tested and the findings are written up as empirical evidence (hypotheses either accepted or rejected) with a measure of statistical significance and a specified confidence level.

An important design in both experiments and surveys detail is the correct identification of the variables under consideration and their nature. Variables are usually described as being independent, dependent or mediating. An independent variable is one that (is likely to) cause the outcome under consideration. That is, tax rate could affect compliance behaviour. Dependent variables are the outcome variables and they depend on the independent variables. For example, the dependent variables in this example could be non compliance and compliance; or unintentional non compliance, intentional non compliance and compliance, and so on. Mediating variables are those that mediate the effect of the independent variable on the dependent variable, but are not expected to be the cause. For example, the age of the person being studied or their level of education could have a mediating effect on the outcome. By stating the relationships to be studied (aligned, of course, to the purpose of the research), the variables can be identified and categorised, thus allowing the researcher to design the experiment or the survey accordingly. At this stage it is often quite helpful (for both researchers and readers) to have a diagrammatic representation of the relationships being tested.

Identifying the relationships to be tested does require careful consideration to ensure that ‘causes’ are not confused with ‘mediating factors’ given that no variable is necessarily fixed in terms of its type. The literature may provide some guidance in this respect. For example, in respect of age, level of education and so on (commonly referred to as the demographics) findings in the tax compliance literature are noticeably mixed and on this basis it is reasonable to conclude that the demographics are mediating variables rather than the ‘cause’ of a particular type of behaviour. It may be possible to control the independent variable (for example, using a fixed tax rate or set tax brackets) to look more closely at the influence of the mediating variables. An alternate view is that it is difficult to control variables such as taxpayers’ attitudes and this may be an inherent design weakness in this methodology.

Selecting populations and then getting access to their data are typically problematic in tax research. How does one identify the population of tax evaders if they are to be surveyed so as to understand what drives their behaviour or the extent of their evasion? Whilst the tax authority may have knowledge of the level of tax evasion, by its very nature it will be imperfect, and regardless, access to this type of data by external researchers is unlikely. What happens in practice in experimental tax research is that university students are typically used as subjects. This in itself may be an inherent weakness and a valid reason for interpreting and generalising experimental results with caution. However, Alm and Jacobson[18] argue that there is no reason to believe that the cognitive processes of students are different to those of ‘real” people. Further, they state that there is now much evidence that the experimental responses of students are seldom different than the responses of other subject pools.[19]

There are a number of types of experimental designs.[20] An experiment means modifying something in a situation, then comparing the outcome with what existed without modification.[21] A full experimental design (in the real world) is very expensive and not always possible.[22] In comparison, a controlled experiment can be a relatively inexpensive means to collect data and has been successfully used in tax research.[23] A controlled experiment needs subjects to be randomly assigned to at least two groups, one being the control group. An alternative approach is to use a quasi experiment where there is no control group nor necessarily random assignment, and commonly pre and post tests are used to measure the effects of a range of treatments (as causes).[24] Typically the results of an experiment can only be discussed in relation to the group of subjects and any more broader generalisations made would need to be suitably qualified. It should also be recognised that the attitudes being tested may be so personal that subjects might be unwilling (due to social undesirability[25]) or unable to disclose them in response to direct questioning.

In the social sciences generally there is the danger to over-interpret, to ignore the other uncontrolled variables that occur in experiments. Stouffer[26] uses an example of comparing the attitudes of two groups of men from the armed services at the same point in time. One group is from the infantry and the other from the air corps. He points out that we cannot know assuredly how much of the difference in attitudes between the two groups can be attributed to the experience in the given branch of service and how much is a function of the attributes of the personnel. True, we can try to rule out various possibilities by matching, comparing men from the two groups on the basis of their age and education for example. But there is all too often a wide-open gate through which other uncontrolled variables can march. ‘One lone study, however well designed, can be a very dangerous thing if it exploited beyond its immediate implications.’[27] Further, Stouffer expresses a ‘central brooding hope that we will have the modesty to recognise the difference between a promising idea and proof.’[28] These cautionary comments are of course equally applicable to both experimental and survey designs.

Surveys are generally described as being structured, semi-structured or unstructured and can be conducted via various means including electronic, telephone, in person or by mail. Structured or semi-structured surveys are generally aligned with a quantitative methodology and they are the focus of the discussion in this part whilst unstructured surveys are discussed in more detail in the following part. A structured or semi-structured survey can be used for collecting both empirical and non-empirical data (usually by means of open-ended questions). It can be a useful tool for testing hypotheses and making statistical generalisations to broader populations. However, as a strategy of inquiry, a survey does have inherent weaknesses. Typically there is much attention given to the sample population, its size, sampling rate and means of selection, and to the issues of bias, response rates (i.e. the reliability of the findings) and external validity (i.e. ability to generalise more broadly) as reflections of the objectivity of the research.[29]

Possibly a more critical issue in survey design is not its objectivity, but the design of the instrument itself. Surveys presume that all the questions and possible answers are known prior to the questions being asked.[30] Further, as Axinn and Pearce note, whilst a key feature of surveys is standardized questions, respondents’ interpretation of the questions is not standardised.[31] Further, they assert that this high level of structure makes it difficult to use a survey to uncover completely new hypotheses.

There are clearly means by which the design of survey instruments can be improved. Whilst pilot and cognitive laboratory testing can be used to some extent to identify and correct misunderstandings,[32] the importance of developing a survey instrument that has clear, unambiguous and useful questions is a critical first step towards improving its internal validity[33] (that is, that the findings have not been affected by the instrument). After all, even when a high level of reliability is achieved, a lack of internal validity must cast doubt on any findings. De Vaus[34]provides the following useful checklist (and extracted advice) for designing surveys which would be readily applicable in the context of tax research:

1. Is the language simple? (i.e. avoid jargon and technical terms)
2. Can the question be shortened? (i.e. the shorter the question the less confusing and ambiguous it will be)
3. Is the question double-barrelled? (e.g. how often do you visit your parents?)
4. Is the question leading?
5. Is the question negative? (e.g. marijuana should not be legalised. Agree/Disagree)
6. Is the respondent likely to have the necessary knowledge?
7. Will the words have the same meaning for everyone?
8. Is there a prestige bias in the question? (e.g. exaggerate income)
9. Is the question ambiguous?
10. Do you need a direct or indirect question?
11. Is the frame of reference for the question sufficiently clear?
12. Does the question artificially create opinions?
13. Is personal or impersonal wording preferable?
    14. Is the question wording unnecessarily detailed or objectionable?

Structured and semi-structured surveys have been described as rather blunt instruments for information gathering. They are powerful in producing statistical generalisations to large populations. They are weak in generating rich understanding of the intricate mechanisms that affect human thought and behaviour. Other methods (typically from the interpretivist paradigm) are preferred for that purposes.[35]

4. Qualitative Methodology

The body of literature on research conducted using qualitative methodology, and on the paradigm itself along with its strategies of inquiry, continues to grow and develop. Whilst much of the work on the philosophy of interpretivist research has been undertaken in the United States and the United Kingdom, there are some excellent texts by Australian authors referred to in this section of the paper.

Liamputtong and Ezzy describe writing a guide book for qualitative research as akin to “trying to write a guide for writing poetry. There are certain rules and conventions, pitfalls to be avoided, and good examples to follow. However, in the end, the best qualitative research depends on the creativity and insights of the researcher themselves. The researcher must find the best way of studying how meanings and interpretations are constructed in their particular substantive research area.” [36] While their text is generally written in the field of health, it is primarily social research and thereby of relevance to taxation.

Within the qualitative methodology, a number of theoretical frameworks exist including ethnography, case study, narrative, phenomenology and grounded theory and each of these (and their associated strategies of inquiry) is discussed herein in more detail.[37] Other theoretical frameworks do exist, such as symbolic interactionism, feminism, postmodernism and hermeneutics, and no doubt more will emerge over time. Given the transitional state of qualitative methodology, Liamputtong and Ezzy make the important point that a qualitative researcher should not assume that the particular theory and research method used in their project will be understood by all other qualitative researchers. That is, if a piece of research is to be meaningfully understood and assessed by other qualitative researchers, the researcher must explicitly state the theoretical tradition and methodological criteria employed.[38] This point is aptly illustrated by the use of ‘case study’ in the literature as both a theoretical framework and also as a strategy of inquiry in the tradition of Yin.[39]

An ethnography is the study of an intact cultural group in a natural setting over a prolonged period of time, primarily by collecting observation data, but can also include data collected by focus groups or in-depth interviews. An ethnography attempts to interpret and present its findings from a cultural perspective and is more so associated with anthropological research than with social research. Given its conditions of conduct, its application to tax research is limited.[40]

A case study framework explores in depth a program, an event, or one or more individuals. Creswell describes a case study as being bound by time and activity, with researchers collecting detailed information using a variety of data collections procedures over a sustained period of time.[41] An example of this type of framework being successfully used in the context of tax research in Australia is the study by Wallschutzky and Gibson on small business compliance costs wherein participating business owners diarised their compliance activities over a twelve month period and underwent in-depth interviews on a regular basis.[42]

Creswell describes narrative research as a theoretical framework in which the researcher studies the lives of individuals and asks one or more of them to provide stories about their lives. Typically, the ‘data’ would be collected by written account or by an in depth interview. This information is then ‘restoried’ by the researcher into a narrative chronology.[43] For example, the lives of former Treasurers or former Taxation Commissioners could be the subject of narrative research. Liamputtong and Ezzy explain that in this approach the researcher typically works with the interview or the person’s biography as a whole, making sense of the story, rather than breaking down the text into discrete parts or collecting information on observations at interview for subsequent analysis.[44]

Phenomenology seeks to identify the ‘essence’ of human experiences concerning a phenomenon, as described by the participants in a study.[45] The emphasis is on gaining an understanding of the situation from the perspective of those who have experienced it.[46] For example, the phenomenon of the cash economy could be studied with this framework and an understanding of what drives people to participate could be gleaned. Other examples in a tax context could be the study of new businesses and how they engage with the tax system, or the study of the impact of the baby bonus on taxpayer’s decisions regarding family, or the impact of a tax audit on taxpayer compliance behaviour. Focus groups, in-depth interviews, semi-structured interviews, or Yin’s case study protocol[47] would all be possible strategies of inquiry in a phenomenology study.

In grounded theory the researcher attempts to derive a general, abstract theory of a process, action or interaction that is ‘grounded’ in the views of participants in a study. Two primary characteristics of this design are the constant comparison of data with emerging categories and theoretical sampling of different groups to maximise the similarities and differences of information.[48] Strauss and Corbin argue that given the way in which grounded theories are constructed, they are likely to offer insight, enhance understanding and provide a meaningful guide to action.[49] The challenge for the researcher is to be open-minded, to listen to and hear what is being said and to interpret it as honestly as possible, always checking and rechecking for other possible interpretations. In the context of tax research, a recent scoping study into the drivers of compliance cost for small businesses in Australia successfully used this framework with data being collected primarily by the conduct of in-depth interviews with a range of businesses and accountants using a combination of theoretical and convenience sampling.[50]

Against this background of theoretical frameworks from the qualitative methodology, the strategies of inquiry now deserve attention. Qualitative research is typically about seeking answers to questions (and sub questions) and not about proving or disproving hypothesis. The strategies of inquiry generally seen in qualitative tax research are in-depth interviews and focus groups and their purpose is to collect what is often described as ‘thick’ data. That is, it is the rich information that the researcher is looking for that doesn’t fit into Likert scales, the data that will help the researcher explore the complexity of the research problem and build an understanding or an interpretation. Observation is also a method of data collection that could be appropriate in tax research, such as examining subjects’ tax returns or record keeping procedures, or the level of stress they exhibit in solving a tax problem.[51]

Within this methodology, in-depth interviews can be semi-structured or unstructured and the interviewer can be passive, empathetic, probe or even provocative. It is essential that the interviewer be skilled and well able to build rapport with interviewees. In practice, this strategy is normally conducted by the individual researcher rather than by an assistant. Questions are not normally constructed or standardised in advance (as in the structured survey of the positivist paradigm), but instead the research has a theme list as a guide to establish the topic at interview. The interview is then shaped according to the experiences of the subject. Questions do need to be open ended and to ebb and flow based on the experiences being told and, in the case of grounded theory, the emerging theories under consideration. Subject to ethical considerations, taping of interviews can be helpful to the researcher but may hinder the frankness of the interviewee.

Typically, the interviewer listens actively, takes copious notes, both of what the interviewee has said and also of the interviewer’s own observations about what was said. Where there is more than one interviewer, it is important that they regularly debrief and discuss what they believe was being said. Whilst there are many software packages (with variations as to the form of data to be input that ideally should be considered before note taking takes place) to help with text analysis, depending on the number of interviews they may or may not be necessary. Given the time required to conduct in in-depth interviews, their numbers may well be limited. If text data is to be analysed manually, it does help strengthen the findings if it is done in a systematic fashion. Liamputtong and Ezzy advocate the use of open coding, axial coding and selective coding as a three-stage methodical and defensible approach to plotting the story.[52]

Open coding, or ‘first run’ is used to identify the dimensions (relationships, events, patterns or themes) that could help conceptualise the organisation of the data overall. Axial coding requires closer examination of the details within one dimension, while selective coding takes a bigger picture view again, but with more of the detail. It may be necessary to work both backwards and forwards through the stages, refining the coding and reconfiguring the framework as required. Miles and Huberman[53] explain that this conceptional organisation of data can be done in a matrix form and this was the approach adopted in the study by McKerchar, Hodgson and Walpole.[54] In this study there were seven matrices refined into one meta-matrix which was framed around five emerging themes. A grounded theory was then constructed for each of these themes on the basis of the data analysis.

Focus groups (or a focused interview) came into being after WW2 and have been used extensively since the early 1980s in many fields, predominantly in public health, but their use is also apparent in tax research.[55] They are regarded as useful when a researcher wishes to explore peoples’ knowledge and experiences, focusing on a narrow range of ideas about and acceptances of new programs and evaluating and solving specific program problems and evaluating health programs.[56] A focus group is not a group interview, but a stimulating group interaction and this can give it a powerful extra dimension as a strategy of inquiry. The setting needs to be relaxed and informal and the participants need to feel uninhibited without any one dominating or withdrawing from the group. Properly managed (in terms of the moderator, the length of time, the nature of the questions, and the recruitment of participants), it can be a very rich source of data. Ideally, participants in a focus group can feel greater confidence in the group setting and can also corroborate or challenge the responses of other members and remind one another of certain phenomena.[57] However, a focus group does not allow the views of any one individual to be fully explored and the group can be led astray by more vocal participants. Similar processes for data collection as described for in-depth interviews would also be appropriate for a focus group strategy. Further, some tax-related topics (such as tax evasion) may be too sensitive for focus group strategies, or may be beyond the scope of the group’s experiences unless purposively chosen (e.g. international taxation).

The qualitative methodology does not lend itself to the making of statistical generalisations consistent with positivism. Instead, its proponents seek to make analytical generalisations or interpretations about a process, rather than its outcomes. They are seeking understanding and explanations rather than a definitive answer about the size and scale of a phenomenon. This is not to say that a researcher in the qualitative methodology should not consider the issues of validity, reliability and sample selection, indeed they should, but the means by which they address these concerns are different to that of the quantitative methodology. In terms of validation, Sarantakos explains that cumulative validation is an appropriate approach, whereby the findings of the study can be validated by various means including the support of other studies. Alternatively, communicative validation can be sought by various means including involving the participants in a subsequent review process, or by employing expert external audits, or by using a Delphi “group of experts” consultative approach.[58]

Reliability (either internal or external) as an issue tends to receive little attention from qualitative researchers. However, it can be addressed to some extent by making greater use of debriefing of interviewers when more than one is used, and/or by peer review. The representativeness of sample selection in the qualitative methodology is generally considered to be irrelevant and unimportant, with Miles and Huberman holding a contrary view.[59] Unimportance is generally inferred as qualitative studies have no real capacity for their findings to be generalised beyond the boundaries of the sample used. However, it is true that the higher the extent of generalisability, the higher the value of the study so the issue is one that warrants further attention. Hence the temptation to overgeneralise and to make conclusions that cannot be substantiated.

Sarantakos does discuss strategies by which generalisability of qualitative findings can be improved, such as conducting multi-site research.[60] Yin’s repeat application of a case study protocol and the study of consistency in emerging patterns is another means by which greater generalisability of findings may be achieved.[61] Drawing conclusions that can be substantiated does require researchers to demonstrate methodological rigour. This can be challenging given the fluidity of the methodology. Herein lies some of the inherent weakness of the qualitative methodology and they need to be recognised. On a more positive note it can be a very rewarding experience for the researcher and indeed a privilege, but one that does come with responsibilities to honour the trust placed in the researcher by those who have willingly shared their experiences for the sake of the cause.[62]

5. Legal Research

Legal research has somewhat lagged behind quantitative and qualitative research when it comes to philosophical paradigms and acceptable conceptual frameworks. Salter and Mason[63] explain that this resistance to methodological discussion in the discipline of law can be attributed to a misconception that legal research is about the technical process and the acquisition of skills (such as identifying relevant case and statute law) rather than about the methodology or different approaches by which the objectives and goals of the researcher can be addressed. They posit that, in the case of student dissertations, they can only be improved by reflecting critically upon, and then justifying explicitly, the appropriateness of their methodologies given their research questions and topic.[64]

In the Australian context, Hutchinson[65] writes that every legal research project is based on an underlying paradigm, but that it is often unarticulated and the research may end up being dysfunctional as a result. She describes the legal research paradigm as a unifying rationale that gives direction and guidance, and notes that success within the discipline tends to be measured within the paradigm boundaries. That is, there is an expectation that legal research is conducted on a methodological basis that is understood by others and can be justified. Clearly the fundamental principles of legal research in respect of making appropriate choices (of questions, methodology and strategies of inquiry) and having alignment throughout the research design are not inconsistent with the expectations of quantitative or qualitative methodologies.

Still, the legal research paradigm is somewhat elusive. The Pearce Committee[66] identified the two typologies of legal research as being doctrinal or non-doctrinal. Doctrinal research is described as the traditional or ‘black letter law’ approach and is typified by the systematic process of identifying, analysing, organising and synthesising statutes, judicial decisions and commentary.[67] It is typically a library-based undertaking, focused on reading and conducting intensive, scholarly analysis.

In contrast, non-doctrinal research is characterised as research ‘about law’ rather than ‘in law’ and employs the methodologies commonly used in other disciplines. It follows that the data used in non-doctrinal research is not limited to the traditional legal sources. The Pearce Committee further divided non-doctrinal research into reform-orientated and theoretical research. Reform-orientated research is designed to accomplish change in the law, and theoretical research is that which fosters a more complete understanding of the conceptual bases of legal principles. It is difficult to identify a philosophical distinction per se in these variations as each appears to have merit and is no doubt more ably suited to a given research purpose. This point is ably demonstrated by Bentley[68] in his thesis that employs theoretical research in its early chapters to gain an understanding of the conceptual bases of the relevant legal rules and principles, followed by doctrinal research to critically evaluate the legal rules and their interrelationship using both induction and deduction. The underlying thread of the thesis is to propose reform by providing recommendations for change, based on critical examination.[69]

Salter and Mason describe the existence of tension or friction between the traditional black letter academic lawyers[70] and the concern of many black-letter academics to the apparent threat posed by socio-legal and other alternate approaches appearing as the Trojan horses. It is undoubtedly more productive to not consider the various methodologies as being engaged in a competition of sorts and thereby mutually exclusive, but as providing opportunities for researchers to design and conduct exemplary research that makes a worthwhile contribution to the body of knowledge. It is encouraging to note that in the Australian context, Hutchinson does make the encouraging assertion that the legal paradigm is changing to a more outward looking focus encompassing interdisciplinary approaches to methodology.[71]

Whether doctrinal and/or non-doctrinal strategies of enquiry are adopted, there are conventions to be followed, or at the very least, rationales to be provided. It is clear that legal research needs to be systematic, purposive and have a robust framework. For example, the analytical framework employed by Walpole[72] in his thesis (using a doctrinal strategy) on the reform of the taxation of goodwill was that of the desirable characteristics of a tax structure as identified by the Meade Committee Report (as the most sound and accessible modern attempt to do so).

6. Mixed Methods

The discussion thus far has focused on the various methodologies as stand-alone approaches, but this is not necessarily the case. Further, there are other paradigms including critical realism, pragmatism, action research (used successfully in education and with potential application in tax research), feminism, queer theory, symbolic interactionism, structuralism, postmodernism and methodological anarchism (in which there are no valid methodological rules, hence ‘anything goes’).[73]

While each strategy of inquiry or typology can be applied in its own right, there is increasing evidence of researchers drawing from more than one paradigm, or using more than one strategy of inquiry from within the same methodology. Each strategy has its strengths and weaknesses and the drive for mixed method research (i.e. using more than one strategy of inquiry to collect data) is to use one strategy to either inform, validate or compensate for the weaknesses of another. In terms of both research design and conduct, mixed method can offer real scope for researchers who are driven by the purpose of the question and not necessarily bound by any one paradigm. It can help fill in some of the gaps, but may not necessarily be superior than a single strategy of inquiry if it is not well designed and conducted.

Multiple methods of data collection require multiple methods of data analysis, and Creswell explains this makes it more important than ever that there is a rationale for the overall design that will purposively and systematically guide the processes.[74] There are key decisions to be made about the order of implementation of the strategies, the priority given to the various strategies, and the point of intersection of the strategies.

The implementation of strategies of inquiry may be sequential or concurrent and serve different purposes (such as to explore, to explain or to corroborate). Collecting data sequentially allows the findings from one strategy to inform another. For example, a focus group could be used to discover the key concerns of tax agents and this could then be used to develop a survey to be conducted with the total population of tax agents so that statistical generalisations could be made (i.e. qualitative methodology used to inform the design of a quantitative methodology). Further, in-depth interviews could be conducted with individual small business owners as to their compliance concerns, followed by focus groups with industry groups to gauge the extent to which findings may be applicable more broadly (i.e. a qualitative methodology used to validate another qualitative methodology). Another example could be to first conduct a large-scale survey about taxpayers attitudes followed by a case study protocol to explore how these attitudes might be formed (i.e. a quantitative methodology followed by qualitative methodology). The possibilities are endless and are not limited to just quantitative and qualitative methodologies. Similar designs could include legal research, for example, a focus group to gauge employers’ reactions to a proposal to proposed legal reforms arising from doctrinal analysis.

While these examples are of sequential applications, concurrent applications can be readily envisaged. For example, a project recently conducted by the Inspector General of Taxation (IGT) into revenue bias in private binding rulings had at least two concurrent strategies of inquiry – one being an evidence-based technical investigation (conducted by the IGT) and the other being an investigation into the perception of bias held by applicants (conducted by Atax).[7] This is an example of a problem being able to be divided into components which do not inform each other, but nevertheless do contribute to the overall solution. Alternatively, the independent findings of concurrent strategies directed at addressing the same problem (or the same component of a problem) can be subsequently compared thereby perhaps achieving a triangulation of findings and improving the validity of the research.

It follows that different strategies may well receive different emphasis within the research, depending on their purpose in the overall design. This is not problematic provided that there is a sound rationale. Similarly, there is a point of intersection of data from the strategies. This can be at the collection phase (such as a survey that includes open-ended questions, or observational and textual data collected at interviews), or at the analysis stage (such as in the thematic approaches of matrix analysis or case study protocols), or in the forming of conclusions and recommendations. Creswell does explain that there does need to be a binding theoretical perspective to mixed method research, whether it be a participatory lens (e.g. small business taxpayer; private binding ruling applicants) or the advocacy lens (e.g. reform of anti-avoidance legislation), as this theoretical perspective helps guide the framework.[75] That is, the possible combinations for mixed method research are almost unlimited.

The key challenge in mixed method design is to develop and conduct the best and most appropriate combination of strategies. In theory it does offer potential to shed deeper light on perplexing problems, in particular, on the nature of causal relationships, but it still may not necessarily provide all the answers given the complexities of human behaviour.[76]

7. Conclusions

Throughout this paper there are a number of consistent messages. The overarching message is that there are many approaches to research and no one methodology is necessarily better than another. Each methodology and each strategy of inquiry has its strengths and weaknesses. However, different methodologies and different strategies are more suited to answering different types of research problems. Further, researchers will have personal preferences for different approaches depending on their ontology and epistemology, whether implicit or explicit.

A researcher must adopt a strong conceptual framework for a piece of work and be prepared to justify that framework to others. The process of justification is more straightforward where the research adopts a philosophical approach that is readily understood by others, and a known strategy of inquiry that fits under the paradigm umbrella. That is, it is an easier pathway if the research can be placed in an existing paradigm and to do this the researcher needs to be familiar with the relevant literature. This does not mean that a researcher cannot develop a new systematic and rigorous framework for the design and conduct of research, but if this is done, then the researcher must be prepared to defend it.

Whichever methodology is adopted, it is fundamental that research has a clear purpose and that there is strong alignment between purpose and design. If the design cannot be tweaked, then the purpose may need to be. Ideally the purpose should drive the design, but in practice the researcher may well have to compromise to ensure that the purpose is achievable given existing constraints (for example, time and/or resources). Time spent on developing a strong and robust design is time well spent, as is time spent on documenting the design and conduct of the research. Attention to detail is paramount.

Further, there are also other important messages albeit at a more applied level. Knowledge claims must be consistent with the strategy of inquiry adopted (or strategies in the case of mixed method) and with the evidence. The researcher needs to take time to reflect on the analysis and to consider alternate interpretations. Modesty needs to be exercised in making claims and in not overstating the contribution made by the research. Give due recognition to the strengths and weaknesses of the research. No design is perfect, and even if it was as close to perfect as can be, things do not always go according to plan. For example, the response rate to a survey may be lower than expected. The researcher needs to explain what has been done to address the problem (the possibility of which should have been considered in the planning phase) or to minimise its effect. It does not necessarily mean the research is ruined, but the researcher may have to consider alternate strategies and/or suitably qualify the findings. Be prepared for the unexpected, or at least, be prepared to find a solution.

Research is almost never conclusive. There are always some nagging questions or areas that emerge but that are realistically beyond the scope of the research. A good researcher will reflect on his or her contribution and identify these areas that need further investigation, leaving a trail for others. The interest of others needs to be stimulated if they are to prove or disprove the proposed theories and continue to contribute to the discipline, or even to create a new one. A good example of this is the pioneering work of the late Cedric Sandford and the recognition by his peers of his creation of compliance costs as an academic discipline.[77] While very few tax researchers will ever be honoured in this way, seeking peer review by presenting and publishing works is an important aspect of the process of developing as a researcher and in encouraging and inspiring others.

---

[∗] Associate Professor of Taxation at the Australian School of Taxation (Atax), University of New South Wales, Sydney, Australia. Email: [e-mail]. An earlier version of this paper was presented at the 20th Annual Conference of the Australasian Tax Teachers Association, University of Tasmania, Hobart, 23-25 January 2008.

[1] Grix, J., 2004, The Foundations of Research, Palgrave, Basingstoke, p.171.

[2] Schwandt, T.A., 2001, Dictionary of Qualitative Inquiry, 2nd edn, Sage, Thousand Oaks.

[3] See note 1, p.98.

[4] See note 1, p.173.

[5] Guba, E. G. and Y. S. Lincoln, 2005, “Paradigmatic Controversies, Contradictions, and Emerging Confluences” in Denzin, N. K. and Y. S. Lincoln (Eds.), The Sage Handbook of Qualitative Research, 3rd edn., Sage, Thousand Oaks, pp.191-215.

[6] Denscombe, M., 2002, Ground Rules for Good Research: A Ten-Point Guide for Social Researchers, Open University Press, Buckingham, pp.21-22.

[7] In this context a methodology refers to the middle ground between philosophical paradigms and discussions of method. See Schwandt at note 2, p.161.

[8] Sarantakos, S., 2005, Social Research, 3rd edn, Palgrave Macmillan, Basingstoke, p.5.

[9] Denzin and Lincoln at note 5, p.3. They discuss the ‘eight moments’ in qualitative research in considerable depth.

[10] Glicken, M.D., 2003, Social Science: A Simple Guide, Pearson Education, Boston.

[11] See note 1, p.85.

[12] Creswell, J., 2003, Research Design, 2nd edn, Sage, Thousand Oaks, p.11.

[13] Axinn, W.G. and L.D. Pearce, 2006, Mixed Method Data Collection Strategies, Cambridge University Press, Cambridge, p.3.

[14] See note 8, p.29.

[15] Banakar, R. and M. Travers (Eds), 2005, Theory and Method in Socio-Legal Research, Hart, Oxford.

[16] See note 1, p.96.

[17] Liamputtong, P. and D. Ezzy, 2005, Qualitative Research Methods, Oxford University Press, Melbourne, p.38.

[18] Alm, J. and S. Jacobson, 2007, “Using Laboratory Experiments in Public Economics”, National Tax Journal, Vol. LX No. 1, pp.129-152.

[19] See note 18 at p.143.

[20] For a comprehensive review of tax experimental design studies see Alm, J., 1991, “A Perspective on the Experimental Analysis of Taxpayer Reporting”, The Accounting Review, Vol. 66 No. 3, pp.557-593.

[21] Neuman, W.L., 2003, Social Research Methods: Qualitative and Quantitative Approaches, 5th edn, Allyn and Bacon, Boston, p.238.

[22] Stouffer, S.A., 1950, “Some observations on study design”, The American Journal of Sociology, Vol. 55 No. 4, pp.355-361.

[23] For example, see Baldry, J., 1987, “Income Tax Evasion and the Tax Schedule: Some Experimental Results”, Public Finance, Vol. 42 No. 3, pp.357-383.

[24] For example see Loo, E.C., 2006, “Tax Knowledge, Tax Structure and Compliance: A Report on a Quasi-Experiment”, New Zealand Journal of Taxation Law and Policy, Vol. 12 No. 2, pp.117-140.

[25] See note 13, p.51.

[26] See note 22, p.357.

[27] See note 22, p.358.

[28] See note 22, p.361.

[29] For more detailed discussion on these issues and the conduct of surveys see for example Fowler, F., 1993, Survey Research Methods, 2nd edn, Sage, Thousand Oaks.

[30] See note 17, p.57.

[31] See note 13, p.4.

[32] See for example, McKerchar, M., 2003, The Impact of Complexity Upon Tax Compliance: A Study of Australian Personal Taxpayers”, Australian Tax Research Foundation, Sydney.

[33] See note 14, p.85.

[34] De Vaus, D.A., 1990, Surveys in Social Research, 2nd edn, Allen and Unwin, Sydney.

[35] Groves, R.M., Fowler Jnr, F.J., Couper, M.P., Lepkowski, E., Singer, E. and R. Tourangeau, 2004, Survey Methodology, Wiley, New York.

[36] See note 17, p.2.

[37] See note 12, pp.9-11.

[38] See note 17, p.29.

[39] Yin, R., 1989, Case Study Research, Sage, Newbury Park CA. For an example of its use in tax research see McKerchar at note 32.

[40] See note 12, p.14.

[41] See note 12, p.15.

[42] Wallschutzky, I.G and B. Gibson, 1993, “Small Business Cost of Compliance”, Australian Tax Forum, Vol. 10 No.4, pp. 511-543.

[43] See note 12, p.15.

[44] See note 17, p.125.

[45] See note 12, p.15.

[46] See note 17, p.19.

[47] See note 39.

[48] See note 12, p.14.

[49] Strauss, A. and J. Corbin, 1998, Basics of Qualitative Research, 2nd edn, Sage, Thousand Oaks CA. p 12.

[50] McKerchar, M., Hodgson, H. and M. Walpole, 2006, “Scoping Study of Small Business Compliance Costs”, Report to the Board of Taxation, Atax, 31 August.

[51] See for example note 32; Woellner, R., Coleman, C., McKerchar, M., Walpole, M. and J. Zetler, 2007, “Can Simplified Legal Drafting Reduce the Psychological Costs of Tax Compliance?: An Australian Perspective”, British Tax Review, No.6, pp.717-733.

[52] See note 17 at p.268.

[53] Miles, M. and A. Huberman, 1994, Qualitative Data Analysis, 2nd edn, Sage, Thousand Oaks.

[54] See note 50.

[55] For example, see Coleman, C. and L. Freeman, “Cultural Foundations of Taxpayer Attitudes to Voluntary Compliance”, Australian Tax Forum, Vol.13 No.3, pp.311-336; Woellner et al at note 51.

[56] See note 17 at p.79.

[57] See note 13 at p.7.

[58] See note 8 at p.86. For an example of the use of a Delphi strategy see Evans, C. 2007, “Unravelling the mysteries of the oracle: using the Delphi methodology to inform the personal tax reform debate in Australia”, eJournal of Tax Research, Vol.5 No.1, pp.105-134.

[59] See note 53.

[60] See note 8, p.98.

[61] See note 39.

[62] See note 17, p.72.

[63] Salter, M. and J. Mason, 2007, Writing Law Dissertations, Pearson Education Ltd, Harlow Essex.

[64] See note 63, p.31.

[65] Hutchinson, T., 2005, Researching and Writing in Law, 2nd edn, Thomson, Sydney.

[66] Pearce, D., Campbell, E. and D. Harding, 1987, Australian Law Schools: A Discipline Assessment for the Commonwealth Tertiary Education Commission, AGPS, Canberra.

[67] See note 66, p.309.

[68] Bentley, D., 2006, “A Model of Taxpayers’ Rights as a Guide to Best Practice in Tax Administration”, PhD thesis, Bond University.

[69] See note 68, pp.6-7.

[70] See note 63, p.35.

[71] See note 65, p.21.

[72] Walpole, M., 2006, “Proposals for the Reform of the Taxation of Goodwill in Australia”, PhD thesis, UNSW.

[73] See note 8, p.31.

[74] See note 12 at p.208.

[75] See note 12, p.213.

[76] See note 13, p.17.

[77] Evans, C. Pope J. and J. Hasseldine, (Eds), 2001, Tax Compliance Costs: A Festschrift for Cedric Sandford, Prospect, Sydney.