Build an AI Patient Facing FHIR® App

Build an AI Patient Facing FHIR® App

A patient wants to connect their medical records to an Artificial Intelligence (AI) to figure out what medical conditions are documented about them. We will use "state of the art" AI with AWS Medical Comprehend.


How we will make it


AWS Medical Comprehend

This is a HIPAA-eligible natural language processing (NLP) service that uses machine learning to extract health data from medical text. No machine learning experience is required.

[I wish I got paid by amazon, but I don't]


Steps to Create an AI AWS Medical Comprehend App with FHIR



  1. [Register an App]
  2. Patient Logs In
  3. Patient Authorizes
  4. App gets list of documents via DocumentReference
  5. [App gets documents via Binary]
  6. [App extracts text from pdf]
  7. App sends text to Jeff Bezos 💸
  8. Render the result

For prior videos on how to register apps

Today we will use SmartHealthIT sandbox, which does not require registration and you don't need Binary to get the document. If you are looking for how to register an app, look at some prior videos such as

Today's Code


To use on your own, create your own repl account and click fork.

You should get something that looks like this after you have forked it


You need to put your own repl url in your own launch.html redirect


Now launch your webapp!


You should see something like this at your own repl url


Click on the link to login like a patient (pre-filled because of smart health it)


Patient approves


You see something like this (after a few seconds)


How does that work?

Launch.html tells FHIR Server - "Hey, I'm launching here, do the OAuth2 thing. Once you are done, send me back to the following url"


App finishes OAuth2, then gets documents about a specific patient and ships them to AWS


AWS Comprehend

I set this up for you already in app.html (please note, I changed the api-key because the billing charge was too much to leave public)


Essentially, create a lambda and hook it into API gateway ... here is the lambda

var AWS = require('aws-sdk');
var comprehendmedical = new AWS.ComprehendMedical();

exports.handler = async (event, context, callback) => {
	var obj = {}
	var params = {
      Text: event.text

	var entities = await comprehendmedical.detectEntitiesV2(params, function (err, data) {
        if (err) { console.log(err, err.stack); }// an error occurred
        else     { obj.out = data; }// successful response

	let response = {
		statusCode: 200,
		body: obj
	return response;


Make sure to have a policy for AWS Medical Comprehend attached to that lambda


App renders results



That's your AI power patient facing FHIR App

Thanks for joining

Learn more at

Extra: Make a Fake Medical Document

Create a Note With DocRef!

(I believe Smart Health IT purges documents each day)