Intranet teams are often faced with shrinking budgets and team sizes, which increase workloads and make it difficult to include observational user research in redesign projects. Instead, many teams survey their employees to inform their redesigns. Any research is better than none, and product suggestions and complaints about intranet issues communicated through feedback surveys can be helpful. However, self-reported data is not enough for a good redesign, and can be misleading: survey responses often represent only a small proportion of employees and may paint an unrealistic picture of the existing strengths and weaknesses in the current design. For example, some of the usable elements of the intranet may go unnoticed and may even end up being changed in favor of fixing issues for the few “squeaky wheels” who complained.
Observational research (including in-person and remote testing and user interviews) is commonly used and recommended to inform intranet redesign. However, in this article, we point out the advantages of a less used but informative research method: field studies.
Some of the main benefits of doing intranet field studies include:
- seeing realistic context and user data
- understanding user journeys (and correcting inaccurate theories)
- building empathy with users
Seeing Realistic Context and User Data
Lab studies isolate participants in artificial settings. Despite the researchers’ best efforts to simulate real-world conditions, sometimes that is not possible. For example, a participant reaching a dead end in one task might say, “When I do this at my workstation, I pull up this document that’s on my desktop and upload that file.” However, you won’t see how she retrieves the file (Does she remember the name? Is it written on a piece of paper by her desk?) and how easy it is to use it. Or, an employee accessing the intranet on the phone, in a busy environment, may have a lot more competing stimuli to deal with than the same employee completing the same task in a lab setting.
A field study provides data about how users divide their time across tasks, physical or contextual limitations that may influence task completion, as well as props and flows that employees may develop in order to optimize their efficiency.
Not only do field studies enable rich observations in realistic contexts, but they also show you how people work with their actual data. In the field or a lab, getting clearance to observe their data largely depends on the sensitivity of the information being processed and on the authorization levels of the research team. If the user’s real data can be observed, in the lab or field, the observations will be more accurate and more insightful than if the research team were to come up with “nearly real” data.
Understanding User Journeys (and Correcting Inaccurate Theories)
Field studies not only help us understand the details of the different activities as people perform them in the wild, but also provide us with a holistic picture of these activities and how they fit in the users’ workflow. They tell us:
- What the actual journey is
- How the employees transition across different channels (digital or physical) to complete their task
- What sources of information the employee relies on
- Where in the journey the employee is having a hard time
- Why the employee is having a hard time and what specific need is not addressed at these points
In a typical work setting, users may engage in activities that the researchers did not anticipate, or they may complete tasks in unexpected ways. Thus, field studies may ultimately build a more accurate representation of workflows within the organization. They can also help identify workflow pain points and needs that are not met due to a mismatch between the design of the existing system, the assumptions of the team building the system, and the mental model that users have of the system.
Users may be able to tell us about some of these pain points. However, their rendition of the problem may be incomplete or inaccurate. For instance, while serving in the Army Reserve, I once volunteered to run a field study intended to improve communication with service members who resided outside a unit’s geographic region. I had initially conducted interviews and surveys with these service members and identified a common complaint: their forms were often rejected due to improper formatting or inaccurate information. Many service members suffered monetary consequences such as paying out of pocket for their own travel expenses; failing to get paid for duty; and getting flagged as delinquent for training when, in fact, they had submitted the paperwork months before. Further, many headquarters personnel grew resentful about these issues and started to hold assumptions that these geographically separated service members “were lazy, didn’t read the forms, and didn’t sign into the shared drive like they were supposed to.”
However, the field study revealed system and form issues, which were the true causes of the problem. These issues were experienced in different ways by multiple users:
- System administrators were forced to deny access to parts of the share drive, to protect against accidental data removal: people would often copy files to their own local or personal folder to “save” them; however, some users inadvertently moved the file out of the shared drive in the process.
- HR administrators at higher echelons wanted to protect themselves against liability, so each level of bureaucracy that processed the form had its own followup document required. This practice resulted in up to three versions of a form, each needed by different people at different levels.
- HR administrators at lower echelons needed to maintain an up-to-date version of the form, but they had no power over the higher-echelon’s digital forms. So, they ended up forwarding any updated forms to service members via email, and otherwise collecting and printing the forms and putting them together in stacks outside each unit’s HR office — that was the only place where you could find a completely updated version of the all the documents.
- Service members assumed that this stack of forms was composed of the same documents on the shared drive (but they were not). Even when using the correct forms, the users rarely had a good frame of reference for how “right” was supposed to look, because of a lack of information governance or “right” example.
The list of issues discovered was very long for this specific study, but it all boiled down to misunderstanding the needs of each type of user, and a lack of visibility of those users.
The solution was ultimately an overhaul of the workflow, stricter information governance, and modifying the documents so they could be used by multiple groups and accessed via a less restrictive system without compromising data security. We probably never would have come up with these solutions if we had not done field studies to truly understand the reality of the users.
Workflow mismatches are frustrating and counterproductive in a typical workspace. In the case above, they affected pay, travel, and personnel-strength metrics. Although in peacetime this information can seem relatively low-stake, in a deployed setting this could result in potentially uninformed or dangerous military decision making. In workspaces such as military, first responder, or healthcare, the mismatch in expectations can result in a difference of life or death.
Many intranet users often do tasks that incorporate inputs from a number of sources. Sometimes a source may be another system. Other times, it may be information found outside of any digital system. Many users will often keep information handy on paper, posted in their workspace, or written in their phone. Information silos may not be visible in a digital system but become more tangible when observed in a workplace. Field studies provide you with a glimpse into the workarounds they’ve created to make their lives easier, and how those workarounds might influence others.
Other research methods also do not as readily reveal human relationships. A spoken request from a colleague is untraceable in a digital system. Participants in our field studies often verbally asked a supervisor or a peer nearby for information as opposed to looking it up online or writing an email. Observing these cases enables researchers not only to understand how relationships among employees affect task performance, but also to gain more insight into workplace dynamics and culture, such as:
- How some employees prefer to interact with each other
- How some words may have different interpretations for different people and user types
- Whether some employees feel isolated from other groups based on the nature of their workspace (e.g., they work alone in the field) or their responsibilities.
These insights can offer ideas for improvements or new functionalities and can ultimately lead to better workplaces.
Building Empathy for Users
Like other observational methods, field studies allow teams to build empathy for users and to step into their shoes—well, at least their workspaces—for a moment. Intranet designers are often intranet users themselves, so the temptation to make unfounded assumptions based on their own use is big. Yet, even in this circumstance, the tenet “You are not your user!” applies. Field studies provide an understanding of the motivations behind certain user behaviors and help teams move from blaming the user to blaming the system.
During a client field study which we ran last year, a nonprofit providing job placements and professional certifications to young adults (“students”) in low-income communities, planned to redesign its intranet portal. One of the biggest pain points noted by the staff was the lack of engagement within one user group: the students. Some students would not respond to job opportunities, and others would not communicate with the organization after they had started their new job. Initially, the perception was that students needed more training on how to use the intranet portal, or that the students were not upholding their agreement to communicate with the organization after they joined the workforce.
However, as the field study progressed, it became clear the students were very responsive and eager to engage, but they did not engage in the intended way – they would call and email rather than using the tool. It also became clear the tool was difficult for many students, because it was built primarily with the staff in mind and then retrofitted for the students to use. Other critical pain points emerged for other roles as well, not just the students, including productivity-halting workflow issues, multiple redundant systems, and a lack of a unified language within the organization. While the problem was initially perceived to be one of educating and persuading the students, it was ultimately much larger than that, influencing other roles in the organization as well.
Identifying the real problem is one of the main reasons to conduct field research. After all, if you solve the wrong problem, it doesn’t matter how well you solve it. A great design of the wrong thing? It’ll still be the wrong thing.
Unlike usability testing, which makes it relatively easy to accommodate observers, field studies often require constraining the number of live observers, due to physical space as well as study effectiveness. Using web conferencing or remote live streaming services can enable more people to watch a field study live. But these tools may require users to install software, can interfere with their activities, and limit observations to on-screen actions. Plus, some users may not be willing to have their activities shared online. (If you do stream the session, ensure that only your team can view it and that the user’s privacy is protected.) In practice, building empathy through field studies is often done indirectly — not through direct observation of users by a large number of team members, but rather by understanding behavior patterns, workflows, motivations, and pain points and sharing those with the rest of the team.
Summary
It can be challenging to run a field study: obtaining the authorization to view real data can be a whole ordeal on its own; and observing people in their real work environment and taking notes is challenging and time-consuming. However, this method can reveal a plethora of undiscovered issues and risks and is well worth the time, money, and effort. No matter what is at stake within the intranets on which you work, a field study with about 10 users is often enough to reveal pain points and to shine a light on big-picture issues. These pain points are incredible insights and opportunities just waiting to be discovered. Once we identify them, addressing them can make a world of difference to our designs.
For more information on intranets, take a look at our Intranet reports.
Share this article: