Undergraduate Research in Computer Science: Lessons, Growth, Advice
Takeaways from my first year of computer science research
During my junior year of college, I began my foray into research by joining a group focused on studying secure processor design. Recently, a full year of painstaking work later, my collaborators and I submitted what will be my first research paper to one of the major conferences in the field.
You read that right — conference, not journal. CS is weird that way. Here’s why, although things might change.
I’m writing this article not only to reflect on my experience but also to share insights with folks considering doing research, or going through this for the first time. This might also be appropriate if you’re wondering what it is like at a high-level, but keep in mind that every research project is going to be different — you’re addressing questions no one knows the answer to, after all!
Insights from my experience
Taking ownership of a research project
One of the decisions I am happy my advisor made was having me contribute to a research project from the beginning. Most often, undergraduates are brought on to assist graduate student mentors in writing code in the middle and late stages of the project. Although this is a great way for them to learn some new skills, it precludes them from being able to influence the direction of a project and contribute to the high-level ideas underlying the work.
I got to play with the problem statement, explore various approaches, and present these ideas, which not only helped me understand the big picture but also helped motivate me to keep making progress.
Developing a background (a.k.a. learning to read papers)
I spent my first few months just learning all the vocabulary and developing my background, just to be able to make sense of all the references that were thrown at me during meetings. I distinctly remember feeling completely lost as I read my first few papers. I was having to look up the meanings of terms in every sentence, not knowing what any of them meant. It was quite frustrating to read those first few papers as they were constant reminders of how little I knew (this feeling hasn’t changed, but I have grown to accept this). Reassuringly, I was told that everyone went through this phase while developing their background in an area, and so I trudged on.
Things became better faster as I honed my strategy of reading papers.
Background and Related Work
The reason research papers are information dense is twofold. Firstly, they are intended to be read by other experts in the field (not laypeople!), who are assumed to possess the background needed to understand the paper off the get-go. Secondly, conferences impose strict page count limits, which results in authors condensing a lot of information into a few pages.However, all hope is not lost. Occasionally, papers will have a background section that introduces referenced material that contains information you need to know to understand the paper. Reading this section can be an incredibly valuable (and efficient way) to internalize prerequisite concepts.
In addition, you will almost certainly find a section on related work. This section exists to ensure that the authors of a work are aware of and have surveyed similar work in the area. You will find them compare the merits and limitations of the current work in relation to a litany of other works. Expect to see a loads of citations. If you are finding it challenging to understand one work, you might benefit from reading prior work addressing a problem, that is cited by the paper. As a rule of thumb, older papers are easier to understand, since they tend to be more fundamental.
In particular, I learned about the three-pass approach to efficiently read papers. This helps you with two things:
- Explore the paper at a level of detail suitable for your needs
Perhaps you figure out in your first pass of the high-level details (e.g. the abstract, introduction, and conclusion) that the paper is not relevant. You can now skip reading the rest and save yourself some time since lower-level details are unlikely to be useful. - Grasp the contributions of a paper, in the context of the problem you are trying to solve
The reason a paper was written is rarely the reason it is read. As a reader of the paper, you might be looking at the work with a different lens than the authors, because you are trying to adapt the idea in another domain or analyze properties of the work not directly addressed by it.
Reading papers is a skill that improves with practice. At this point, I certainly feel much more comfortable picking up a fresh paper in the field and quickly digesting its essence than I was when I began this journey.
One of the interesting things I learned to do and subsequently did many times, was conducting a literature review.
What is a literature review?
A focused study of related research publications with the aim of understanding the current state of knowledge in a particular field, often with the purpose of identifying limitations and potential areas of improvement.
Here are a few things I learned while doing reviewing the literature:
- Finding the right papers — this process is all about identifying important keywords that will help you guide your search. Google Scholar is great for this. One rule-of-thumb is to find a highly-cited piece of work in the specific area, and then skim through its references as well as papers that cited it. Another way to do this is to skim programs of top conferences in the area over the past few years, looking at sections that pertain to your problem.
- Managing references — I recommend using a tool such as Mendeley or Zotero. These tools help you organize everything you’ve read, highlight important sections, write notes, export citations, and also sync across devices.
- Figuring out what to cite — you certainly won’t cite every paper you read (but it is still useful to have gone through them!). You might realize that an improved version of an idea is what you really want, or that there is an earlier work that originated a particular line of inquiry. This is why it’s valuable to explore the related work.
Navigating research software
Unless you are working in theoretical computer science or in an area whose research methods are impractical to reproduce (such as CS education, usable security, and human-computer interaction), you will need to reproduce a research result, build a system from the ground up, or use a research application to analyze data.
Having spent some time in the industry, I noticed some immediately obvious differences in industry vis-à-vis research software, with the caveat that a lot of these probably don’t apply to projects that have been popularized in industry, like PyTorch, Kafka, and Kerberos:
- Availability — software developed as implementations and evaluations of a research idea may be nigh impossible to obtain. Artifact evaluation is not a mainstay at all conferences. You can try emailing the authors, but there’s no guarantee they will be willing and/or able to hand it to you, assuming they choose to respond to your email in the first place.
- Documentation — most often you will need to read the code yourself to understand exactly how something is being implemented, and if it covers scenarios that are important to you. This is where persistence really pays off, as being able to re-use open source code can save you a lot of time you would have otherwise spent reimplementing a baseline system.
- Testing — often, you will use software that ignores industry developer best practices, style guides, and concerningly, has no tests. This means that it will be challenging to modify, and easy to break, which can lead to weeks being wasted squashing unexpected bugs.
- Developer community — depending on the popularity of the paper and how recent it is, there may be fewer than 20 people in the world who’ve developed or used that software, which means sites like StackOverflow are not going to be particularly helpful.
While navigating research software, I recommend filling in the gaps that you find missing. This means creating your own documentation that should help not only yourself but also future collaborators. It means writing test programs that provide a sanity check of behavior.
As an increasing number of conferences push for reproducibility, getting any artifacts (benchmarks (software or hardware), proofs, and survey results created as part of your research) evaluated increases the validity of your contributions.
For code, consider spending some time (perhaps after a paper deadline) tidying up the code, including examples or test programs of how to use it, and potentially putting it up online (such as on GitHub) for others to use. Depending on your availability, responding to questions from the community (i.e. people wanting to use the artifact) could be your way to pay it forward!
Managing your advisor
Nick Feamster and Alex Gray have some great advice on managing one’s advisor(s). Check out their blog How to Do Great Research for quality coverage of life in academia and survival skills for graduate school.
The bottom line is that professors (tenure-track, teaching faculty, or otherwise) are busy people, and you need to work around this. Here is what I’ve learned:
- Agree on expectations early, and set up regular meetings
When a professor agrees to advise your undergraduate research or independent study, they are making an investment in you. Make sure that both of you are on the same page with respect to time commitments, and the expected progress. Setting up regular meetings is a great way to maintain accountability and consistent progress.
If, for whatever reason, you are feeling overwhelmed by work, or are swamped with coursework, communicate communicate communicate! Do not disappear off the face of the Earth and ignore emails/messages— not only does this reflect negatively on you, it also makes professors more hesitant to take on other undergraduates in the future! - Ask for what you need
Professors are excellent resources for a number of things, but only if you ask for their help! Helping their students succeed is literally what they signed up for with their job. However, they cannot read your mind. Whether you are looking for advice, funding, job opportunities, recommendation letters, or a short meeting with them, take the first step by asking (courteously). Having said that, be respectful of their time — if you are requesting something, ensure you make the ask well in advance. Be prepared to follow-up numerous times to ensure it happens. - Emails — best served short, to-the-point, and with important information first
You want to make it as easy as possible for a professor to respond to your email. Ideally, lead with the information you want to share, or your request. Chances are, if it’s too verbose, you may not even get a reply, which leads me to my next point. Matt Might has some excellent advice on how (or whether) to send and receive email. - If you don’t get a response in a few days, send a gentle nudge
For people receiving dozens of emails a day, it is very easy to miss a particular email or forget to respond by getting preoccupied with something else. A polite follow-up reminding them of the previous email is a great way to put it on top of their queue, and signals that you are invested in the conversation.
Advice on undergraduate research
Try it out
Doing research has been one of the highlights of my undergraduate experience, and I highly recommend people try it, even if only for a semester.
Here are some benefits:
- Learning graduate-level material in an area of your interest
Did you ever want to explore that one section of a class, that there wasn’t enough time for, in greater detail? This is how you can do it, without the burden of exams and grades to worry about. - Developing technical reading, writing, and presentation skills — all of these are incredibly valuable in academia and in the industry.
- Considering graduate school? Most graduate schools require 3 letters of recommendation — instead of just being able to say that you did well in class (DWIC), they will be able to write a much more personalized and glowing letter of all the awesome things you did while working with them.
There is a lot of information out there on how to get involved in undergraduate research, so I will be pithy. The key components are:
- Identify a problem or field you’re interested in.
- Find a professor (or graduate student) who will agree to advise you.
- Jump in, and expect to learn a lot!
Note that 1 and 2 can be done in the opposite order. Particularly if you’re at the beginning of your undergraduate career, and unless you’re really picky about a field, having a supportive advisor whose working style matches you will likely lead to a much better experience than working on an interesting problem but with someone you find it difficult to work with.
Attend talks, and keep up with literature
If your department offers seminars, distinguished lectures, or research talks of any kind, definitely take advantage of them! Morning job talks at my university’s CS department were accompanied by coffee, tea, and snacks; meanwhile, our weekly Security and Privacy Research at Illinois (SPRAI) seminar came with Jimmy John’s sandwiches! These are ways to keep in touch with what is happening in other areas of CS, and
This year, I was lucky enough to be able to attend USENIX Security 2020 and the Symposium of Usable Privacy and Security (SOUPS) 2020, where I got to listen to some highly interesting talks learn from conversations with experts. Conferences are a great way to keep in touch with the rest of the academic community and can be a great way to start new research collaborations.
Despite not being able to show up to conferences, I do try to skim a sample of papers from sessions that I’m interested in, which helps me stay informed about bleeding-edge work that could potentially inform my current work. This is most easily done for open-access venues (e.g. all USENIX conferences) and if you are affiliated with a university that subscribes to digital libraries such as the ACM Digital Library, or IEEE Explore. Many conferences upload all their talks online, which can be a useful resource. Watching these talks can be a helpful primer to the area and specific work, before you go off reading the paper (if you are interested in the details).
Persistence is key
Research is about learning things about the world no one knows about, your advisor included. Expect it to be challenging. You will end up at dead-ends sooner or later. That is a natural part of the process. I myself switched projects twice before we landed on the most recent one that we’ve brought to closure. As you get more experienced, you will get a better sense of what to try next, when you get negative results. Eventually, the goal is to be able to independently conduct the whole process!
Final thoughts
I have enjoyed my research and look forward to doing even more. Working with professors is a unique experience quite different from being lectured to as part of a large class. My only regret is not getting involved earlier.
Resources
Here are some resources I recommend to get involved with research and improve your strategy.
A special shout-out to my advisor Chris Fletcher and collaborator Jose Rodrigo Sanchez Vicarte for their valuable mentorship!
With thanks to Reed Oei and Nathan Ju for valuable edits and feedback.
