J. MARESCAUX imagin, robotics and A.I. : the 3 weapons to achieve a smart surgery

Good morning everybody. I think for me it's really a pleasure to participate to this 31th edition of the digestive system surgery that you organize in Rome

and to thank Professor Palazzini because it's always a big and very important event.

Sorry that it is just in visio conference today without anybody in a big room as usually in Roma.

I want to speak about new technology that we develop in IRCAD, and especially to speak

about how important it is for surgeries, the development of imaging technology, robotics,

and artificial intelligence.

And we will see that these three topics are totally linked together because the robot

of tomorrow will be a robot integrating image and integrating artificial intelligence.

So always, when I start my lectures, I explain that I had not the idea to create this institute

in Strasbourg and the other one, but I was so lucky to attend in 1991, so you imagine

30 years ago, an exceptional lecturer from Rick Satara that everybody knows, he's a

colonel of the U.S. Army, he's a surgeon, he's a visionary, he's an exceptional man.

And you imagine that in 1991, during a conference where he spoke about medicine, we were going to switch from industrial age to information age, he spoke about robotics, virtual reality, augmented reality, artificial intelligence.

You imagine, that was 30 years ago, nothing exists, but I was so impressed, I came back

in Strasbourg, and we have developed an institute totally dedicated to this new technology.

And that is the concept of research and development of IRCAD.

Everybody knows IRCAD for the training session we have, but we have also a big team of researchers,

and the topics we work on is augmented surgery.

And augmented surgery, if you speak to a surgeon and you ask how we can imagine to improve

our art, the first answer is, I want to see more than what my eyes can normally see, and

that is augmented eye or virtual reality and especially augmented reality.

Second one is, I want to improve my hand, and that is augmented hand and that is robotics,

not only for laparoscopic surgery, but also for flexible endoscopy,

for the development of endoluminal surgery, which is more and more important.

And finally, I want to improve my intelligence.

I want to improve my strategy.

I want to improve my adaptation during an operation, before an operation, after an operation,

and that is artificial intelligence.

If we start by the first one, which is augmented eye, we have developed in IRCAD, since now

more than 20 years, a lot of software which are able today to transform a 2D image of

a CT scan in a 3D image of virtual reality.

That is more than the volume rendering of a 3D CT scan, because the image can be an

interactive one.

So it's a specific 3D reconstruction.

We have created a spin-off of IRCAD, whose name is Visible Patient.

And due to the development of artificial intelligence,

a reconstruction which took two years ago two hours now can be done in 10 seconds.

So here, for example, for neck surgery,

you can analyze and see all the details of the anatomy, the arteries, the veins.

You see here we have this big artery behind the oesophagus.

We know that it is what we call an arterial usoria, that we have a non-normality of the

trajectory of the nerve.

So it's so important, and we develop that for all the different organs.

Here is a very nice example, because we show that during all our advanced course of hepatobiliary

surgeries, the cholangiocarcinoma, all the experts coming from Japan, Europe, United

United States, they say this cholangiocarcinoma is on the right part of the liver, so it's

planned to do a right hepatectomy, and when we have the 3D reconstruction, you will see

in transparency that due to an abnormality of the absence of medial vein, it is not on

the right side, but on the left side, and this kind of software, you can use, you see

virtual clip, for example. So we put the virtual clip on one vein and we know immediately what is

vascularization of the liver, that means what is the territory of vascularization of the liver.

So it's a perfect analysis before an operation. We can do that also for pediatric surgery.

It is a big amartoma on a young baby of two months old. Very difficult to know where to

to start the operation, but with the 3D reconstruction, we can navigate inside the tumor.

Immediately, you will see that it was possible for the pediatric surgeon to see that the

two arteries going to the tumor were going from the right hepatic artery, and so it has

been very easy to start the operation and to control these two arteries.

So we have a lot of different examples for different organs.

So that is the first step.

The first step is the planning.

The second one is the simulation.

Here are three metastases of the liver.

For the two on the left side, we can propose a tris-segmentectomy.

For the other one, we propose an ablation, cryoablation or radiofrequency ablation.

And you see that we can take the virtual optic.

we know exactly what is the best way to be sure that we go just in the center of the tumors.

So it is a perfect simulation.

It's totally interactive.

We can take, for example, the virtual optic.

We go inside, and you see that on the left part of the window,

we see exactly in transparency what we are going to see the day of the operation.

So that is a preoperative simulation.

And we know that with the increasing power of the computer, very soon it will be more

and more realistic.

Finally, augmented reality, virtually made, real image given by the video.

What is augmented reality?

It is a fusion of these both images.

And if we have a fusion of both images, we see in transparency, and that is the concept

of augmented reality.

It's not so simple.

We started nearly 20 years ago

and we published first

in Germany in 2004.

At the beginning it was only

possible to work on

retroperitoneal organs

because they don't move.

So we had a very nice paper

concerning 50

adrenalectomy.

You see how it works.

We have the adrenal gland.

We see in 3D, and during the operation, you see that when we dissect the external part of the vena cava,

we ask the computer engineers to put in transparency the virtual image,

and so we see very well the two main veins, and when we dissect the renal vein,

we see in transparency in the fat the superior polar artery of the kidney

that absolutely we need to preserve, because if not, it could lead to post-operative hypertension.

So that is a concept of augmented reality.

But, as I said, it is not so simple.

Why? The concept was initiated by neurosurgeons and orthopedic surgeons, I think more than 20 years ago.

But for them it was very simple, because it was, you know, a rigid deformation for a rigid

operation, because you know that for a bone it doesn't move.

For neurosurgeons, you know that when they have the CT scan or the MRI of the patient

in the operating room, the first thing they do is to fix the skull of the patient.

And in this case, you have exactly the superior position, because nothing moves.

In the real life of our speciality, digestive surgery, or vascular surgery, or urology, or gynecology, it's not the same.

Because in real life, it's not rigid deformation, it is a non-rigid deformation.

And we have a lot of problems to solve.

First of all, if we want this superior position, we have a first problem.

is that the position of the patient when he has the CT scan or the MRI is not the same

than the position he has during the operation.

Because during the operation he can have a lateral decubitus, he can have a prone decubitus,

and normally the image is not done like that.

So the engineers of IRCA, they have developed a really fantastic thing because with a very

cheap R, B, G, D camera, you see that they can do something like the external mold of

the patient, and if we have the external mold of the patient, we can have a perfect superimposition

of the muscle and the skin.

So the first challenge is solved.

You see, and you are going to see very well, that in real time, during the operation, we

We have progressively the superimposition of the skin and the muscle, so the external part of the patient.

But you see also that we have not the new position of the organ due to the new position of the patient.

And that is another thing that we have to solve.

We have to solve the movement due to the bracing of the patient.

You see the dynamic MRI.

We have a mobilization of the liver of more than 5 centimeters.

And this challenge has also been solved by the engineers of IRCAD.

And you look how they put some landmark on the sternum of the patient.

And they have developed some predictive algorithm.

And you see that due to the mobilization of the position of the sternum, they can calculate the mobilization of the liver or another organ.

So that is the second challenge, which is solved.

But it is still more and more complex because the first thing that we do when we operate is to put a retractor, to put a grasper.

and you see that immediately.

Here it's an experimental, in the experimental lab,

on a pig you see that we have a total modification of the volumes,

the position of the organ,

and if we have a mobilization of an organ,

we have also a different position of the different arteries, veins, etc.

So that is complex.

And that was a challenge that nobody solved in the world.

We discussed a lot with the team of engineers and computer science people of John Hopkins.

It was exactly the same limit.

And it's the reason why we decided, that was ten years ago, to create another institute just faced to EARCLAD.

And to say, we need to have in the operative room all the systems of imaging technologies

technologies to inform the computer of the reposition of the patient, of the organ, etc.

And with a big deal with Siemens, you see that that is a 3D, it was more than one year

to try to imagine how to do, and after that it was built in one year, and finally we have

really some hybrid operating room with CT scan, MRI, ZIGO, OR, PHENO, and 4-day ultrasonography.

So that's this institute, just phased to IRCAD with this possibility.

How it works?

For complex hepatic surgery, for example, you see that we have intraoperative CT scan,

so we have the arterial phase, we have the venous phase.

But because we want also to have the biliary tract anatomy, you see that with MOI, in the

same operating room, we have the biliary tract anatomy.

And during the operation, with this machine whose name is ZIGO, or now FENO, we have in

one second 360 degrees, and so we have an image of CT scan.

But because we are in an operating room, we need to have very low radiation.

If we have low radiation, we have not a good quality of image.

But these machines are so smart, so intelligent, that they take into account immediately the good quality of the CT scan you had,

the good quality of the MRI you have, and you have this possibility of good 3D image.

Finally, I want to speak about the augmented hand, which is a robotic surgery.

I like to show that because it was some fun.

It was done in 2001.

It was the first remote surgery performed in 2001 that we have published in Nature.

Why is it interesting?

Because you imagine 20, not 20 years, but yes, nearly 20 years ago, we had done the 5G.

Today, you know that we look a lot of trials in China, especially they try to do 50, 100 kilometers long remote surgery on an animal.

But at this period, it was not possible.

So we had used ATM lines of the transcontinental cable of France Telecom, and you imagine that

we had a delay of 131 millisecond, that means a delay as of 5G today, 20 years after.

It was fine, it was not a surgical challenge because the operation was a very simple coalescist

but it was a technical challenge of information age, and it was the first time we were able to prove

that one day telemedicine will be applied to telesurgery with how to share the knowledge,

the intelligence, but also the gesture.

So that was, for me, a very important moment, and that was a publication in Nature.

We work a lot on how to integrate the image in the robot system.

So here we have just plugged the 3D reconstruction of a patient.

It was a very, very severe rectal cancer that is operated here by Professor Melani, who

is the director of IRCAD in Brazil, and you see that it is like a GPS in a car.

So we have the 3D image, and when the surgeon is looking inside the master part of the robot,

you see that he is able to see the normal image given by the camera, but also the visual image.

And this picture is very important, because you see the camera, the normal image, you see just fat and blood.

It is impossible to know that you are really in the good plan of the mesorectal excision.

On the left, you see that with the 3D image, we have the mesorectum, we have the arteries,

we have the uterus, and we have the vein and the lymph nodes.

So you have just to understand that if one day, which is really what I think, one day

surgery will be automatic as the evolution of aeronautics, as the evolution of smart

smart car, we cannot imagine that if augmented reality system is not solved.

Because imagine that the left image is put in transparency in the right image.

That means the machine will be able to understand the different steps of the operation.

So that means you can teach the machine, and if you can teach the machine, one day it will

be automatic.

We have worked a lot on the application of robotics for flexible endoscopy, why?

Because we have seen that the gastroenterologists work without any concept of triangulation.

They just work with two instruments going out of the flexible endoscope in a parallel

way, and we know that it is not possible to operate like that.

So we have developed a scarch torsondoscope, what we have called an ubiscope, and you see two flexible instruments at the tip of a normal flexible with possibility of triangulation, and so it is possible to move, to do a suture, etc.

But we have seen that even with this kind of triangulation, it's not so easy.

And something important is that for all alimentary tract surgery, in the next 10, 20 years, it

will be more performed by endo-liminal way than laparoscopic way.

And so we need to develop this kind of instrumentation, and we need to imagine also robotic system.

stem. So we have worked with a team of the University of Strasbourg, which is famous

in robotic surgery, and you will see it very interesting because for an ESD, endoscopic

submucosal dissection, you see on the left a gastroenterologist with an experience of

more than 1,000 ESD, and on the right, a surgeon with absolutely no experience of flexible

endoscopy? Absolutely no. And you see that on the right, it is a real operation. Why?

Because it is performed by a surgeon with a very nice plan of dissection. It is smooth.

It is really an operation, and that is not the case for the left window. So that is certainly

the next step of robotic surgery will be flexible endoscopy.

I want to finish by augmented brain.

Augmented brain is how we can imagine to put artificial intelligence in the operating room.

We had a very nice project during the last three years,

how to imagine to set in the operating room a control tower like in a airport.

And today we know that it is possible and very simple to acquire all the data of the pre-intraoperative 2D and 3D image, the external camera, the laparoscopic camera, the physiological signals given by the anesthesiologist, and all that are storage.

and that is certainly a way for application of intelligence artificial and surgery

because we follow the patient until he is going back to work.

I want one word to say that that is impossible to do just in IRCAD Strasbourg

and certainly you know that since more than 15 years we have a globalization of the concept of IRCAD

not only for the training but also for research.

So today the training in Strasbourg is like that, after the live surgery, we have a lot of on-sum courses in the lab with 20 tables.

We train for flexible endoscopy because we know that this is more and more important.

That is done also with Professor Costamagna from Roma.

And here you see also the platform, that was the first platform of robotic surgery, today

with 13 1-3 robots of Intuitive Surgical, and it is certainly the most important site

in Europe for robotic training.

And now, because we know that it is more and more important to do robotics training than

laparoscopic training, we just finished, in one week it will be finished, a new building

for only dedicated to robotic research, robotic application, and robotic training.

One word about the website, free of charge, so you can be connected and you will see all

the new operations, new medical devices for minimal invasive surgery, more than 430,000

active members, translated in seven languages, I repeat, totally free of charge.

So it's the reason why we have every year more and more active members.

And finally, globalization of the concept, especially for research also.

We start in France.

After that, you see in 2008 in Taiwan, two in Brazil, Lebanon, Rwanda, and China,

who are going to be finished in the next year.

That is the center in Taiwan with a lot of good researchers.

And in Taiwan, they have really talent for technological transfer, something that we

have perhaps less in Europe.

So that is the center in Taiwan, in Brazil, in Sao Paulo state, in Rio de Janeiro.

That is something that we have with United Health, which is a big American company of

private insurance.

that is Hôpital Français du Levant in Lebanon,

and that is a center in Africa, not a big one,

it will be the biggest one in Rwanda.

And something interesting is that it will be very important

for the development of all this technology,

because in Kigali, in Rwanda,

they have a subsidiary of Carnegie and Mellon University of Pittsburgh,

which is the number one in computer science and artificial intelligence.

So they train a lot of Brian brains in Africa, and we extend our team of researchers, and

we are going to hire approximately between 30 and 40 engineers in Rwanda.

And finally, the next project will be finished in China in 2002.

So that is IRCA China in Wuxi, near Shanghai.

It is a complex of four big hospitals, of 700 each, just dedicated to minimally invasive surgery, all the different specialities.

So, just 30 minutes exactly. Thank you very much for your attention.

And if you have any questions, I don't know how it works, that would be absolutely with pleasure.

Thank you, Professor, for your outstanding and visionary presentation.

I don't know if I can turn to you any question, because it's not what it's meaning for this

Congress.

I thank you on behalf of Professor Palastini for your presentation, for your participation

to this Congress.

Thank you.

Thank you very much.

Thank you very much.

It was really a pleasure.

And have a very good day, continue a very nice online Congress, and really congratulations

for all you do in Walmart since more than 30 years now. Have a very good day.

Goodbye. Goodbye. Thank you again.