The Center For Health Design
Why does this study matter?
“Let’s do a POE.” Seems simple, right? Design professionals get the potential value of post-occupancy evaluation, but they often find that there is zero time after construction is complete to create a POE tool and go through the evaluation process. Even if they can carve out the time, they struggle to find standardized POE instruments and processes that will produce results that have any applicability to other projects. And if they do find a standardized tool, which can be good for generalizability, it usually means forfeiting the benefits of a customized tool designed to evaluate how well their project met the organization’s specific goals and objectives. Can standardized tools that have customizable features provide a balanced solution to this evaluation conundrum?
How was the study done?
Alitzer and colleagues gathered an interdisciplinary team (an architect, four facility personnel, a clinician, a health systems admin student, and an engineer) and set out to pilot test a POE process. They used a POE tool focused on EBD goals specific to medical/surgical patient rooms, developed by The Center for Health Design with support from the ASID Foundation. They customized the tool by mapping the healthcare organization’s ten original guiding principles on to the tool’s 23 EBD goals. The tool includes an assessment aid to help the raters determine their rating for each item. Five guiding principles matched up with goals in four overarching categories outlined in the tool. The facilities planning leadership set an internal standard that at least 75% of the items would need to be scored a 4 or 5 – or “top box” - on the Likert-scale for them to call it a “passing” score. Twenty-three audits were completed by eight auditors in three patient rooms.
So what do we learn from the study?
The room design missed the mark according to the organization’s internal target threshold. Sixteen of the 23 items (69.9%) received “top box” scores. The areas that fell short included EBD goals around quality of care and patient experience, worker safety and effectiveness, and organizational performance. More specifically, the rooms received lower scores related to flooring to prevent injury, patient control of the environment, and change-readiness to meet future needs. Overall hand sanitation scores were low as well.
The team learned that doing a POE for the first time is challenging. They realized more training to get everyone familiar with the evaluation protocol would be valuable in the future.
Can we say the results are definitive?
These results are more about the journey than the destination. This tool is just one way they can better understand some aspects of the design’s success. They also recognize the challenges around trying a new process, and that in the future they aim to provide more training. They mention that in the future they would include a more interdisciplinary team. Having external auditors would also help with getting objective findings, as bias is inevitable when auditors have a personal investment.
While the scores fell below the internal target threshold in certain areas, we are left to wonder “why?” For instance, what was it about the design of the room that did not enable patient control? This is a great opportunity for design teams to pair the tool with qualitative methods, like interviews, to follow up and ask about the reason for certain scores.
What’s the takeaway?
Post-occupancy evaluation is tricky. How do you know if a project was successful? Where do you start? What tool(s) should you use?
This pilot study was a first stab at a new process for a group of people who had not used a POE tool before. They faced a variety of challenges, but were able to balance the benefit of standardization with customization by mapping the EBD goal in the tool onto their organization’s guiding principles.
The team should be applauded for sharing their learning process, warts and all. While the patient room design did not meet their internal target threshold, the process provides the team with an opportunity to carry their learnings forward into future projects. The articles shows what happens when you go back to evaluate a project at the end, rather than having a clear plan in place to prioritize the link between design and measurable outcomes from the outset.
We can learn so much from this type of article to move the field forward towards better evaluation methods. There may not be easy answers to the POE puzzle, but it adds to the important conversation about evaluating our designs to see if they pass muster (or not).
Altizer, Z., Canar, W. J., Redemske, D., Fullam, F., & Lamont, M. (2019). Utilization of a Standardized Post-Occupancy Evaluation to Assess the Guiding Principles of a Major Academic Medical Center. HERD: Health Environments Research & Design Journal, in press. https://doi.org/10.1177/1937586718820712
Our slidecasts are an outcome of the popular Research Matters presentations at the annual Healthcare Design Expo & Conference. Our research team picks papers that have some significance to the healthcare design community and distill the study down into a 5-minute summary of how the study was done, what was learned, the limitations and the takeaway. The slidecasts bring research to you in digestible format. Just five minutes, and you’ll know more.
As a part of The Center's research team, Melissa focuses her efforts on grant-funded research initiatives, as well as supporting ongoing work of The Center, such as the Affiliate+ program, the Knowledge Repository, and the Evidence Based Design Accreditation and Certification (EDAC) program. She examines the impact of the built environment on health through the development of conceptual models that link design and outcomes and by evaluating the effects of evidence-based design innovations and by disseminating research findings through tools, presentations, and publications.
Melissa was previously a senior design researcher for Davis Partnership Architects and prior to that she served as a researcher with the OMNI Institute and the Center for Children, Youth & Environments at the University of Colorado. With a background in both architecture and social science, Melissa describes her research style as “boots on the ground with a knack for rigor.”
Melissa graduated from Cornell University with a Masters of Science, Applied Research in Environmental Psychology and received her Bachelors of Environmental Design, Emphasis in Architecture from University of Colorado, Boulder.