O2011: What Leads to a 5-star Rating?: A Sentiment Analysis Comparing Online Reviews of Surgical versus Non-surgical Pediatricians

Background: Physician review websites are a heavily utilized patient tool for publicly finding, rating, and reviewing pediatricians. Natural language processing such as sentiment analysis provides a comprehensive approach to better understand the nuances of patient perception. While online reviews are becoming a more commonly used resource for patients, minimal research has investigated if patients and their families value different traits in surgical vs. non-surgical pediatricians. This study implements a machine learning approach to compare how specific patient sentiments correspond to positive and negative experiences in online reviews of pediatric surgeons and non-surgeons.

Methods: Online written reviews and star-ratings of physicians belonging to the American Academy of Pediatrics and/or the Pediatric Orthopedic Society of North America were obtained from Healthgrades.com. Sentiment analysis was used to obtain compound scores for physicians with ≥1 review. Sentiment scores range from -1 to +1 indicating how qualitatively positive or negative a review is. Pediatric surgeons (n=913 yielding 4282 reviews) and non-surgeons (n=1860 yielding 5619 reviews) were analyzed separately. Word frequency and multiple logistic regression analyses were performed on both cohorts, and inferential statistics were utilized.

Results: Females made up 20.9% of surgeons and 53.5% of non-surgeons. Male surgeons received higher average star scores than female surgeons (p < 0.01) but there was no difference between sexes in non-surgeon scores (p=0.47). Regardless of cohort, younger physicians had higher sentiment (surgeon p=0.04, non-surgeon p< 0.01) and star scores (surgeon p< 0.01, non-surgeon p< 0.01). (Table 1) For both groups, reviews that mentioned physician attributes such as “comfortable'' were more likely to be positive, while “rude” and “unprofessional” decreased these odds. Additionally, while “confident” increased the odds of positive reviews solely for surgeons, “kind” and “warm” only increased odds for non-surgeons. Ancillary characteristics regarding “front-desk” decreased odds of positive reviews for non-surgeons, while “wait” lowered these odds for both groups. “Pain-free” increased the odds of positive reviews for surgeons alone (Table 2).

Conclusion: Both non-surgeons and surgeons who were younger, communicated effectively, and curated an efficient office environment were more likely to receive positive reviews. While most traits and factors seemed to impart a similar influence on reviews for surgical and non-surgical pediatricians alike, confidence was weighted more positively for surgeons while interpersonal characteristics such as warmth and kindness were more highly valued in non-surgeons. Notably, significant differences in star scores between male and female surgeons may show that implicit gender biases play a larger role in how patients review and perceive surgeons compared to non-surgeons. To our knowledge, this is the first study utilizing machine learning to objectively analyze online physician reviews for pediatric surgeons and non-surgeons. These outcomes can advise these pediatricians on specific behavioral and office qualities that patients and their families prioritize when rating and recommending their physician online.