Selecting a good thesis problem is nearly 70% of the battle in completing a Ph.D degree. Considering that as one begins graduate school one really does not know what to work on, it becomes extremely important to develop a sense of what “good problems” are. After being a graduate student, working in industry and being a former academic, I have worked with many colleagues and students in the research enterprise. As I reflect on the experience, a key topic that comes up is what makes a worthwhile problem to pursue. In this essay, I share some thoughts on the same.
In the context of graduate school, simplistic/self-centric criteria for problem defintion are factors like: a) anything that I will get a degree for is a good problem, b) a good problem is one that has some funding (or funding potential), or c)what my advisor/mentors tell me. Though these are good rules-of-thumb to start the winnowing process, a bit of extra effort helps in the long run. The reason is problem selection is a step that is repeated potentially many times in one’s career. For example, after graduate school, the issue of defining a problem (as a grant proposal if you are in academia) that garners one’s attention and resources becomes more important and directs one’s choice of career. Assuming that one is of the entrepreneurial kind, the choice criteria may be as simple as any idea that may make money is worth my attention or somebody is willing to fund (even a funding agency such as the NSF or NIH) or even R&D groups within a company..! Anyways, in my opinion, the key skill that is “formally” never paid that much attention to in graduate school is how to pick a good problem or a more viable variant of the same, which is: how to make an ill-defined problem solvable/tangible. This is essential so that one can then know enough to judge how much work is essential for a Ph.D and can have enough objective criteria to judge his or her own work (as they mature as a researcher). At least in my humble experience, without imbibing such criteria, many colleagues of mine often have struggled to know what their work is worth and have usually ended up doing more than necessary in a variety of contexts. Anyways, it is important to realize as a graduate student that you are in school to learn the process of research and most problems will not be handed down to you like text book problems at the end of a chapter. The hard part of research is crystallizing the problem such that a viable solution presents itself. Spending some thought on the types of problems is essential, as the rest of your professional life, you will be constantly choosing problems to solve and get paid for the same (as I suggested earlier). In this essay, I outline some of the key features of problems make them worthwhile to solve. These are abstract criteria that one can use to generate/evaluate/modify/frame problems and sketch potential approaches to solving such problems. Though there are a number of resources (books, videos etc.) on creativity, problem solving etc, what suits a given individual’s basic “problem-solving” paradigm and world-view may be different. Hence, I do not believe, one can prescribe generally applicable suggestions such as: “For problem type Y, looking for solution X, do method Z.” However, it is possible to identify some key features of problem Y, that guides ones choice amongst a number of methods Z. Given this it may be possible in the least to generate different types of solutions X that are worthwhile to refine into a final working solution. Solvable problems “Y” have some key features that I discuss further below. For example, an engineering thesis problem, could be framed as:
- Y – Design optimization of load-bearing I-Frame structures
- X – CAD-based solution
- Z – propose a new mesh type, new optimization algorithm, interleave simulation + optimzation
General Classes of Problems
However, before discussing the features of Y specifically, I want to note the following generic characteristics of worthwhile problems in general. They usually can be classified into the following major bins:
1) Things that we want to know about the material/physical world and things in it. What are they? How do they interact ? Where did they come from? How do they work? What happens in different contexts etc. These are the sciences – natural and social sciences (including people, organizations and others).
2) Things that we (homo-sapiens) have artificially assembled – both real and virtual. Examples include things like physical devices, the web, economic systems, financial markets, corporations, plays, movies etc. These are the applied sciences, life sciences, engineering and technology.
It is important to note that in reality there may be major interplay in any problem of interest between the two classes mentioned above but usually one viewpoint tends to dominate the issue at heart. For example, identification of disease-triggering genes is a biological problem (belonging to the first class), but should it be patented for commercial purposes is a legal/moral/economic issue belonging to the second class. A physical device has to obey all natural laws and is bound by them, whatever we may want to do(engineer) that device for economic benefit. A classic example is that CPU speeds have plateaued at 4Ghz and one cannot pack more transistors without melting the chip. We seem to have reached an upper-bound on Moore’s law and are developing multi/many core chips to gain computational speed.
What are the features of a good “thesis” problem?
The features of interest for a problem for both classes discussed above (in no particular order) include:
1. Definable: Is the problem “well-definable” ? that is : Is one able to clearly articulate the following: why is it a problem? By solving it, what does one gain or understand new ? If a solution is developed for the problem, is it a one-off or answers an open question in the research community? The “clarity” of the language in which the problem is defined and how a possible solution may look like gives a sense of how definable a problem is. Problems usually build on a common “models” of the phenomena of interest – that is described in a shared language by the community of researchers working on the problem. Furthermore, one should be able to define the “frame of reference” of the problem – Is it related to an individual entity or a group of entities ? Is it generalizable from an individual to a collection or from a collection to an individual.
2. Data available or acquirable: A problem should be describable in terms of “real world data – data before the solution is applied and potentially how the data may look after the solution is developed. The key step here is that realworld data situates the problem and “operationalizes” the solution in terms of before/after. If the data is not available, it must be acquirable with currently available instrumentation/data collection methods, else first new data collection methods need to be developed. Many inventions in the natural sciences have been driven by the need to develop instrumentation to just observe the world. In the context of a thesis, this data should be acquirable/available within a reasonable time-frame to complete the thesis. Given the empirical basis of the scientific method, establishing a tangible relevance to real world data is essential.
3. Solving the problem adds to current knowledge: Solutions to the problem may be fundamental or applied. For example, it may be the identification of a new component of a phenomena or suggesting a new mechanism for how something operates or a new tool that does some previously known operation better or understand something deeper etc. The key is that in the context of “science” – the solution suggests something “new” that was not known before. It either adds or corrects some mistaken notion of the past. In the context of engineering, the solution provides a new way of doing current things, extends the applicablity of what is known, generalizes some phenomena, or does it better etc. Solutions to a worthwhile problem usually lead to newer problems with fundamental impact.
4. Nature of the solution: A key aspect of a worthwhile problem – is “what is the tangible nature” of the solution? Is it a description of a phenomena, is it a mathematical or computational model ? Is it an experiment (or set of experiments) that establish a phenomena ? Is it a summary of a data analysis ? Is it an algorithm ? Is it a physical device (a model of its behavior or validation of a behavior of device, its applicability, extending its features etc) ? Is it the development of a data collection? Is it an analysis/reorganization of the past data? Developing a sense of the final deliverable is essential to keep the effort focused over the long time-period of a thesis. Note that a thesis may have one or more of these as deliverables.
5. Current relevance of the problem: Usually, for most problems, a scientific community is collectively in-charge of developing a solution. Further, most communities have current open problems of interest with a collective assessment of the “worthwhile” features list above. Usually a survey of a field will describe these worthwhile problems in broad strokes. The problem you work on must be relevant to the community, along some dimension not only when you start working on it, but when you get to the solution (3 years down) and possibly into the future. Such dimensions include: a) solution provides insight into a current problem, b) solution is a stepping stone in the context of a bigger problem, c) It is an incremental step in a previous approach that was not pursued to its fullest extent etc. It is your responsibility to find these dimensions of relevance. If it is an old problem, you need to identify why it is relevant and why the community needs to pay attention to it ? What is the new wrinkle ? If it is a problem of the future, you need to define how the current trends may lead there, and why a solution to that futuristic problem has an impact now and is relevant to be paid attention to now.
6. Resource/skill requirements: Primarily, one needs to develop a sense of how long it would take to solve a problem – is it a thesis time-frame problem (3-5 years) or a lifelong objective ? This is not an easy question to answer and keeping an open mind is essential to scale down your objective as you understand more of the problem. Further, a sense of how much past effort has gone into the problem provides a sense of its complexity and resource requirement. It is important to note, most problems of interest have been addressed by somebody, sometime before. That is why the whole enterprise is called “re-search” and just not search. Some problems may not be solvable as they are poorly defined or too tightly defined. Further, some problems may not be a single person effort but require teams of people with multiple fields of expertise. Further, some questions may require “special equipment”, building which may take a life time. Examples of such trends are well-established in the natural sciences wherein there is clear division between a theorist and an experimentalist (especially in physics) and then collective efforts are organized in building experimental tools such as the LHC or sending experiments into space etc. One needs to be able to assess these issues as early as possible while embarking on a thesis to get a sense of what is doable or not-doable within a 3-4 year time period.
7. Applicable “methodologies”: Based on the nature of the problem, a sense of its solution and the availability of data – a choice of “methodologies” may be applicable (more on this below). One should be able to identify these methodologies as early as possible to make progress. Part of the thesis process involves iterating through these methodologies and picking the right ones to resolve the issue at hand. Developing a thesis proposal is an effort to perform this exploratory process and develop a sense of “doability” of developing a solution for the problem.
8. Audience for solution/thesis: Relevant problems have an audience/community looking for a solution for different types. These could be academic and/or industry audiences. If a problem has an audience it is publishable in the appropriate outlets – including conferences, journals and technical/scientific outlets. Some solutions may drive the formation of startups and the like. Identifying and understanding the audience is essential to a) fostering the judgement criteria for good research within oneself, b) keeping one self motivated as one pursues the problem (that there is some body who may benefit from your effort) and c) also developing a sense of self (and ownership about the problem/solution) in the community at large.
9. Benchmarking: The problem and solution has to be benchmarkable in the sense of a) What has been done before and how far have we come towards a “reasonable solution” to say that the problem is solved? b) What are other researchers doing?. Benchmarking qualifies and quantifies the scale/complexity of the problem and in turn the “goodness” of your solution. It identifies new problems to be focused on so that others may continue further work on the same.
10. Impact: Finally, a key criteria is the “impact” of the problem/solution – How it permeates the general approach to that phenomena/understanding in the world. Usually, descriptive solutions have a lower impact than “predictive ” solutions (More on this below). Part of the impact factor – is a sense of how fundamental the issue is at hand and does it address the larger objective of the field as a whole? Can the solution/insight be “embedded” in a wide-range of other related areas delivering an order-of-magnitude impact in models, tools, techniques etc. Though qualitatively/quantitatively assessing impact is difficult, developing a sense is essential. Further, from a personal viewpoint, the whole effort has to be self-rewarding, independent of the community.
These features outlined above are contextualized by the choice of tools/techniques to resolve a problem. Our education and research training teaches us a variety of techniques that may be used to solve a problem and most of us end up being good toolsmiths with siloed views of which technique works best in what context rather than focusing on the problem/solution. A good example of this is the slow acceptance of computation techniques in the proof of the Four-colour problem. So just to give a flavor of available tools and techniques in general, I provide a brief overview. Readers are advised to review the following references for further information.
What are the available methods/frameworks to solve a problem?
In the rest of the discussion, I use the term “Science” in its most general form to include the basic natural/social sciences and all their applied fields – in creation/management and operation of devices/systems of realworld value. Most sciences go through atleast two phases (of knowledge evolution) – an initial descriptive phase of the phenomena of interest followed by a more through analytic, data-driven model-based phase with increasing levels of predictive power. I will briefly outline the descriptive approaches first followed by the more analytical approach.
1. Basic case-based, scenario-based descriptions wherein the phenomena is described in humanly perceived terms. Many of these descriptions may be supported by quasi-logical causal models and these descriptions are primarily attempts at “framing” the problem.
2. Descriptive phenomena descriptions may be further refined by taxonomical categorization of the components of the phenomena of interest. Isolating potential components within an overall framework provides focus for the research effort.
3. Data driven Hypothesis- Experimentation – Validation approach- This is the classical technique – aka the scientific method, wherein the phenomena of interest is observed in terms of different attributes. Experiments may be defined to isolate exogenous and endogenous (that is input/output/control factors) for an individual or for an aggregate system. Hypothesis formation is driven by an exogenous framework positing a sequence of interactions based on a philosphical world view.
4. Analytical model development with “predictive” ability – The experimental technique supports the development of an abstract model with mechanistic rules that enable prediction. Predictions of the model than may be validated via experimentation. Some abstract models are mathematical in nature, some are physical/geometrical in nature, some are just “functional” rules etc. Steps 4 and 3 may be utilized in a generate and test mode to address a range of problems. Atleast one cycle of 3 and 4 is essential to say something useful in the context of any problem. It is is also important to note that “mathematics” is one language (although very powerful) to codify an abstract model. The language has its own set of “inferential” rules that allow prediction. These proof methods and their capabilities are an object of study in their own right along with additional constructs in the language of mathematics. Statistical techniques are a subset of mathematical methods that along with Step 3 provide the ability to build models about groups of entities or establish properties of an individual with high-levels of confidence. These techniques form the basis of a large number of topics/phenomena wherein there are large numbers of entities such as molecules in gases to people in an economy. Mathematics by itself has developed a tradition and set of self-consistent techniques such as proof strategies to develop consistent theories.
5. Computational modeling and simulation – Assuming that we have at least reached Step 4 on a problem of interest, one can exploit computers to “simulate” the phenomena of interest and study various interactions. By combining it with Step 3, the range of possible experiments one can do is increased. However, establishing validity of these simulation models is essential to draw any inferences that may apply to the real world. Furthermore, advances in “mathematics” and “computation” are constantly broadening the horizons of what we may study/simulate.
Steps 1 and 2 are primarily used in the descriptive phase followed by 3,4 and 5 in the analytical phase. That’s it! No magic here. Humanity has spent the last 400 years repeating primarily Steps 1,2,3 and 4 in ever increasing cycles and Step 5 has made its advent in the last 60+ years. Parts of these methods in different contexts has been practised in all the ancient civilizations providing us the western and oriental approaches, though for now – the western approach dominates current thought. All methods are primarily variants or combinations of one or more of the above. So as a graduate student, as you search for a problem, you need to have a sense of what methodology(s) may be used to develop a solution (and some of this is dictated by your training).
What is the relationship between the aforementioned methodologies and
the characteristics of the problem outlined earlier ?
The methods mentioned and problem characteristics outlined earlier form the
scaffolding on which modern science has evolved as discussed in Kuhn’s book, The Structure of Scientific Revolution and other studies in the same genre. Most fields of modern science (natural and economic) have emerged from the basic field of “Natural Philosophy” as it used to be called nearly five centuries ago. Furthermore, the interplay between real world applications and underlying theory is a constant ongoing process that leads to new shoots emerging in the tree of knowledge. However, there have been some key trends that are being addressed in the current climate of science: These include:
1) Questioning the “reductionist” philosophy: The basic approach that has been successful so far has one underlying idea: breaking things up into smaller pieces that when understood properly can explain the behavior of the whole. Most experiments have focused on decomposing the problem (or subject) such that it can be experimented with in isolation and understood. This paradigm is slowly giving way to
a “systemic” approach, where one studies topics of interest holistically. Approaches as to how to do this are still being figured out and is unclear how to go about it. A classic example is modern biological systems – one talks about pathways of interaction but it is still unclear as to how do pathways interact in general.
2) Trends towards “Interdisciplinary work”: Modern problems seem to require multiple skills to address/resolve adequately. As a good researcher one needs to be skilled in at least two fields minimally to make any viable progress. Working on inter-disciplinary problems requires you to be well-versed in the modalities of the different fields and atleast an expert in one so that the collaborative effort can be fruitful. Tools,ideas and paradigms from other fields may have to be adopted and their applicability evaluated in a given field.
3) Preference for only positive results: As a researcher, one realizes via unpublished manuscripts that no field rewards negative results with the same weightage as positive results. If you say something negative, it behooves you to find something positive to establish about the problem. Negative solutions as stand alone contributions are not that well rewarded at large. Though this may change slowly, being comfortable with negative results is essential to maturing as a good researcher. One may not be able to publish it, but in a competitive world, it is something that may be put you on the right track and prevent more missteps in the future.
4) The role of the computer: Computers are playing a critical role in increasing the speed and scale at which science/engineering research is being performed. It has become an important exploratory tool enabling a wide range of activities in the research enterprise – from gathering information to planning experiments, to controlling experiments, analyzing data and finally presenting data. It is important to develop a level of comfort with it at many levels and learn “programming” in addition to the 3 Rs – reading, writing and arithmetic. Without such an understanding of computing, one is limited. Programming is easy and most individuals (what ever the field they may be in) can pick it up. We do it everyday as we “program” (a.k.a plan) our lives on a daily basis.
5) Communication: Finally, communicating what your research is all about is essential from the external viewpoint of gaining recognition, acceptance from your thesis committee, funding, getting feedback from your peers and such. Science and technology have become part of everyday life and it is a hugely collaborative enterprise. To be in the mix is essential and the key is to be in control when you want to get into the mix and when you want to be a free radical pursuing your own thought. Working on communication skills with patience is key to improving one’s own self-confidence and being relaxed in one’s own skin as you take on the research issue at hand. The acceptance of your idea has to go through a Darwinian process of evolution as you compete with many other near similar ideas in a highly noisy process!
Strategy for identifying a good thesis problem
Finally after all the above background discussion, it is important to summarize the approaches to generating potential problems that can be evaluated if they meet the criteria outlined earlier. These include:
1) Question current assumptions at different levels of detail about the phenomena of interest – Are current explanations correct ? Are the assumptions relevant and complete? Just because a problem has industrial/real world use – does not mean folks understand the theory behind it! (Remember, life used to go on before we knew Newton’s laws!)
2) Try repeating some work (from papers and such) that interests you and see if you find flaws
3) Try cooking up a quick experiment or perform a thought experiment if things make sense.
4) Try to identify phenomena that current theory/experiments cannot explain or if they do, what detail is missing, and what is being addressed..
5) Try changing the assumptions of how things work and mentally simulate how things may work.
6) Try exploiting a tool in a new context or adding a new capability to a current tool/technique.
7) There is always the reliable – read papers in your field, surveys and read the “originals” – people who are credited with initiating the first ideas – those always suggest some unexplored lines of investigation.
The key is to get started on some line of thought as soon as possible. It is important to be comfortable with things going wrong. Actually many things can go wrong from running out of funding (because the funding agency does not think the problem is interesting enough) to the development of a new trend. Some of these you can control, some you cannot but as an apprentice researcher it is important to avoid fads and buzz. These are things for your advisor to deal with, not you. Going through this generate and test cycle for identifying a problem regularly is essential in developing your sense as a researcher. Usually there are two choices: ) In some case advisors lay out all the steps and you execute and graduate or b) in many cases, advisors give you the freedom and you spend the time figuring it out. In the short run, a) is a good strategy but you flounder later because you cannot avoid this experience when you are on your own. Alternatively, approach b) is a high pressure approach but benefits you holistically in the long run.
Finally, What are the minimal requirements for a quality thesis ?
For a Master’s thesis, it is essential to address just one of the steps in the methods outlined earlier. But that step has to be done thoroughly with clear identification of what existed, what was done, what was inferred etc. Types of MS projects include:
a) Collect scenario descriptions – for example, some case studies on epidemiological analyses.
b) Organize material – previous research, results, approaches in a taxonomy, identify what is the next set of problems and why. If possible suggest/sketch the components of a solution.
c) Perform an experiment to validate an hypothesis, perform data analysis alone to support/invalidate an hypothesis, design a new experiment etc., develop a new device by prototyping etc.
d) Develop a model, based on available data – the model may be mathematical or otherwise, Establish the validity of the model – experimentally by doing new experiments, comparing to already available data or by “computational simulation”
e) Build a computational model, establish the algorithm, evaluate the quality of an algorithm etc.
A good PhD thesis spans steps a),b),c) and d) or a),b),c),e) with d) being done by somebody else or a),b),e),c). The key point is: the guts of PhD thesis focuses on the interplay between c), d) and e) in the context of a problem. PhDs in applied mathematics topics also address the above, whereas pure mathematical approaches focus on internally self-consistent theories. PhDs in CS focus on establishing mathematical properties of computational constructs or validating properties of computational artifacts by building and experiment with these artifacts. PhDs in Engineering focus on modeling real world systems and establishing their properties of interest, whereas PhDs in the natural sciences focus on establishing “natural mechanisms”. PhDs in the social sciences study large-scale man-made systems which interplay with the natural systems. Though these are broad brush strokes, the key point to realize is each problem may require a contextualized approach.
Before we wrap up, it is at least important to know what makes a bad thesis. There are some features such as a) weak or non-existent research/topic/problem genealogy and attribution (as no problem stands alone), b) Bad data gathering practises and awareness of issues that can go wrong, c) Weak models without clear logical basis of what is it that is NEW on some dimension, d) Too inter-disciplinary and too many buzzwords – good inter-disciplinary work delineates perspectives of each field clearly and identifies the solution in those contexts, and e) no generalizability or validation with a real world context. Avoiding these is essential by defining your problem and work crisply and clearly.
Though a vast topic, hopefully this brief essay provides the interested reader an overview what good thesis work entails and some signposts on the road to be travelled.