Idea quantity
The number of ideas that employees submitted in the Idea Portal was lower with hybrid work. This can be seen in Fig. 2, which plots the number of ideas per employee per month over our sample period after removing time trends. Figure 2a plots the unweighted number (the number of ideas an employee submitted, possibly with other employees as coauthors, in a given month). Figure 2b plots the weighted number of ideas (the number of ideas, each divided by the number of authors of that idea, that an employee submitted in a given month).
Average number of ideas submitted per employee per month after removing the linear and seasonal time trends (y-axis normalized: 0 is WFO mean). The vertical bars indicate changes in the work mode (WFO, WFH, hybrid).
The precise estimates of Eq. 1 can be found in Table 1, which reports WFH and hybrid work effect estimates relative to WFO, the base category. WFH did not significantly change the number of ideas submitted by an employee in a constant time window, relative to work from the office (Table 1). This is true for all four idea quantity measures we use: (i) NumIdeas, the number of ideas an employee submitted (possibly with other employees as coauthors) in a given month; (ii) NumWeightedIdeas, the number of ideas, each divided by the number of authors of that idea, that an employee submitted in a given month; (iii) NumIdeasMS3, a three month moving sum of NumIdeas for each employee; (iv) NumWeightedIdeasMS3, a three month moving sum of NumWeightedIdeas with (iii) and (iv) only defined if all three months are part of the same work mode. Table B.1 in Supplement B provides summary statistics for these variables. Note also that focusing on the specification in column (1) of Table 1 we have more than \(95\%\) power to detect an effect size equal to the Hybrid effect or an effect size equal to half of the Hybrid effect at the 5 percent level.
Hence, when it comes to the quantity of ideas, the often-expressed worry that WFH hampers innovation does not find support in our data. However, during hybrid work there was a significant reduction in the number of ideas, relative to both WFO and WFH. Again, this is true for all four idea quantity measures and hence a robust finding. The magnitude of the change in idea quantity during hybrid work is meaningful, since the base rate is small. According to summary Table B.1, employees generated 0.009 ideas per month during WFO on average. This means it takes an employee about 111 months, or a bit more than 9 years, to generate an idea. During hybrid work, employees generated 0.007 ideas per month on average, which corresponds to 143 months or just below 12 years for one idea. These numbers correspond to a drop of 22% of ideas per employee per month in the switch from WFO to hybrid work.
The findings of a zero WFH effect and a negative hybrid work effect are robust to various changes in the statistical model specification and data preparation. Table C.3 estimates the same regressions as in Table 1, except with a quadratic rather than linear time trend. Table C.4 estimates the same regressions as in Table 1, but drops the top 0.1% of outcomes to investigate the sensitivity of the results to outliers. Table C.5 demonstrates that we also find a significantly negative hybrid effect with binary innovation outcomes (i.e., whether an employee submitted at least one idea) in OLS and logit specifications. We also find a neutral WFH effect, hence the results here are robust when using only the extensive margin of innovation rather than counts as in this section. All these tables can be found in Supplementary Material C.
Finally, since we only observe the date of idea submission in the Idea Portal, but not the date of idea generation, it is possible that ideas have been generated weeks before, so we sometimes attribute them to the wrong work mode. To test the robustness regarding lagged reporting of ideas, we re-estimate the average WFH and hybrid work effects with the outcome variables lagged by one or two months in Table C. 6 in appendix C. The hybrid work effect remains significantly negative in all specifications.
These estimates suggest that it is not so important for innovation whether the workforce works from home or from the office, but it is important that they are consistent. Perhaps counter-intuitively, the hybrid work effect is worse than a convex combination of the WFO and WFH outcomes.
A potential explanation for why hybrid might be worse for innovation than both WFO and WFH is costs of coordination and communication. If everyone is in the office, it is easy to talk to colleagues, and meetings can be spontaneous and in person; “watercooler” conversations can take place. Those conversations can lead to the generation of new ideas, but they can also provide feedback (positive or negative) which might spur refinement or reformulation. If everyone is at home, then similarly all are in the same chat rooms and video meetings, and using the same modes of online communication. It is possible in this case to establish substitute channels of communication. However, under hybrid, some employees are in the office and some are at home, and at varying times throughout the workday. Office employees might talk amongst themselves in person, whereas remote employees talk online. Moreover, scheduling a conversation may be more difficult in hybrid mode, relative to both WFO and WFH. These are additional barriers for the team. Getting everyone to talk is harder due to the different modes of communication. Because of these coordination and communication issues with hybrid work, innovation may suffer.
To evaluate this hypothesis, we constructed a measure of the extent to which members of the same team adapt similar hybrid work practices. For each employee, we measured the number of days in the office each month, and then variation in this measure across team members. For example, some teams continued to WFH exclusively, while some were much closer to full WFO, while others were somewhere in between.
Table 2 compares teams with a higher variation in office presence with those that have a lower variation in office presence. Indeed, the former have a significantly worse hybrid effect on the quantity of ideas than employees in teams with low variation in office presence. Hence, teams that are more scattered between office and home innovate less during hybrid (relative to WFO), compared to teams that are less scattered. The interaction effect is significantly negative for all of our four idea quantity measures, which is strong evidence in favor of our conjecture that coordination on a communication channel for informal chats within the team is important. This can explain why teams that are more scattered between office and home are less innovative. That said, the hybrid effect is negative on average even in teams with no variation in office presence; i.e., in teams that either fully WFH or fully WFO. (Among teams with a standard deviation of at most 1 in terms of employee days in the office per team, 89% are fully remote.) Several explanations are possible for the negative effect in well-coordinated teams. One possibility is that substitute communication channels are not being established as rigorously as under WFH, which could be a reason why innovation suffers even if an entire team ends up working from home. But it is also possible that the missing contact to other teams has negative impacts on innovation, in line with the idea that individuals who bridge different teams are often successful innovators13.
We conducted several robustness analyses. In Supplement C, Table C. 9 shows that these results are robust to including an interaction between hybrid and the group mean in office attendance, in addition to the group SD as before. Hence, the lower hybrid effect really is due to more variation in office presence, not more (or less) office presence overall. Moreover, Table C. 10 repeats the analysis of Table 2 using the minutes that each employee spent in the office that month, rather than the number of days they were in the office, to compute the office attendance variation measure. The results are very similar.
Idea quality
While the quantity of ideas is important for innovation, so is the quality of those ideas. A better idea might generate more profit for the firm or more value to the client. Table 3 displays estimates of the WFH and hybrid effect on three measures of idea quality: (i) “IdeaAccepted” indicates whether or not a suggested idea was accepted for implementation; (ii) “ClientShared” indicates whether an idea was shared with a client; (iii) “ClientApproval” indicated whether an idea received a good rating of 3 or 4 (on a 1-4 scale) by the client. As in the quantity regressions, we control for seasonal as well as linear time trends, and we include author-team fixed effects. The sample used includes only ideas where internal review is finished, so these ideas are either accepted or rejected. Hence, informally, the estimates we get are the difference in quality between an idea submitted by the same set of employees during WFH and an idea submitted during WFO, and similarly for hybrid work vs WFO.
In WFH the quality of submitted ideas is lower than in WFO. In Table 3, for all three quality measures, the sign of the estimated WFH effect is negative. The probability of accepting a suggested idea is 6.7 percentage points lower for ideas submitted during WFH, compared to WFO ideas (Column 1). However, this difference is not statistically significant. The probability of sharing the idea with the client is 9 percentage points lower for ideas submitted during WFH, compared to WFO ideas (Column 2). This difference is both economically and statistically significant. The probability of receiving a high client rating is about 18 percentage points lower for ideas submitted during WFH, compared to WFO ideas. This is a large effect, which is also statistically significantly different from zero.
While the sign of the hybrid work effect is negative for all measures, none of the differences are statistically significant. The regressions in Table 3 use only ideas from months where more than 50% of ideas have been internally reviewed, in order to avoid a potential bias if better ideas are reviewed faster. In a simple regression of the months to a review decision on IdeaAccepted (not displayed), every additional month is estimated to reduce the acceptance probability by 1.6 percentage points (\(t=\)− 10.34, SEs clustered on author-team level). Since there was less time for review for ideas submitted during hybrid work, this is the work mode that loses observations first as the review rate threshold is increased. At a review rate of above 50%, as in Table 3, only the last 3 months of hybrid drop out of the sample, and WFH as well as WFO retain all months. This means that, if the “fast review selection effect” is not removed due to conditioning on a sufficient review rate, hybrid is favored by the selection effect. Therefore, if anything, the selection goes against WFO and favors hybrid, so the hybrid coefficient in Table 3 may overestimate the real effect.
This discussion raises the question of robustness of results to choice of review rate threshold. Figure C.2 in Supplement C.2 plots the coefficients of the WFH and the Hybrid dummies depending on the review rate threshold, estimated in regressions as in Table 3 but varying the threshold. Since the review rates do not reach 70% for any month during the WFH and hybrid work modes, we cannot estimate these coefficients for a review rate of 70%, hence the upper bound in the figure is 65%. We chose a lower bound of 40% for the review rate, which drops only a single hybrid work month, and so barely corrects for the “fast review selection effect.”
Figure C.2 in Supplement C.2 shows that the WFH coefficient—for outcome IdeaAccepted—is negative for all review rate thresholds and statistically insignificant for all but one review rate threshold (at 60%). The hybrid coefficient is negative and statistically insignificant for all review thresholds. For ClientShared, the WFH coefficient is generally negative (except at the highest review rate), and significantly negative at 45% and 50%. The Hybrid coefficient is generally close to zero and statistically not different from zero for all review rates. For ClientApproval, the WFH coefficient is generally negative, and significantly negative at 45% and 50%. The Hybrid coefficient is generally negative and statistically insignificant.
In summary, the robustness analysis in Figure C.2 demonstrates that there is never a significantly positive WFH or hybrid work effect on idea quality, irrespective of the chosen review rate threshold or quality measure. For almost all specifications, the signs of the effects are negative, the Hybrid effect is always statistically insignificant, and the WFH effect is sometimes significantly negative. Our conclusion is that the Hybrid effect on quality is statistically zero, and the WFH effect on quality is non-positive. That is, for WFH, the evidence is divided between significantly negative and insignificant estimates, with the latter in the majority. As neither a conclusion of no WFH effect nor a conclusion of a negative WFH effect are completely robust, we conclude that WFH has a non-positive effect on idea quality. The zero hybrid effect is remarkable, because idea quantity declines as we have seen above. The evidence in this section shows that it is not the case that the worst ideas are discarded first in this process. Instead, the decrease in innovation seems to affect the good ideas at least as much as the bad ones.
In Supplement C, we get the same results and very similar estimates when using a quadratic rather than linear time trend (Table C. 11). Moreover, we show that these conclusions remain if we assume the idea submission dates were one or two months earlier, to allow for the possibility that ideas were conceived earlier and possibly under a different work mode. These estimates are displayed in Table C. 12. Finally, Supplement Table C. 13 shows heterogeneous effects for the three quality measures.
Did the type of ideas change with work mode? The fraction of process improvement ideas dropped significantly by about 10 percentage points during WFH, relative to WFO, a massive effect (the fraction in the entire sample over all work modes is 26.7%). Both WFH and hybrid significantly increased the fraction of cost reduction ideas compared to WFO, by about 9 percentage points (the hybrid effect is significantly different from zero only at the 10% level). Again, these are large effects; the cost optimization category has a fraction of 19.6% in the entire sample. Last, pure WFH produced more technical solutions than hybrid work.
The change in idea composition does not explain the negative effect of WFH on quality, though. We re-run the regression to estimate the average WFH and hybrid effects in Table 3 (in Supplement C), but include the indicators for the idea categories, to get the average WFH and hybrid effect when holding the idea composition constant. These regressions are displayed in Table C. 14. The qualitative results from before remain: the WFH effect is significantly negative for the client-related idea quality measures, while the hybrid effect remains statistically zero. Thus, the change in idea composition does not explain these negative WFH effects, and in fact, the negative point estimates get slightly more extreme after controlling for the idea composition. Hence, there is something else about WFH that reduces some aspects of idea quality.
link