Jacques Marescaux Robotic and Artificial Intelligence

It is in the top 100 worldwide universities in the world, in the ranking of Shanghai,

and that is certainly a reason why we can attract so many researchers and industrial partners.

I always like to start my lecture by this video, just to say that never I had the idea to create

and all the technology against cancer but in 1991 i attended an exceptional lecture of

this man rick satala he's a surgeon colonel of the u.s army and we imagine that in 1991 he explains

the next step of surgery especially saying that the it will be the strength of internet that was

1991 internet arrived in 1992 the strengths of robotic surgery virtual reality augmented reality

applied to surgery and oncological surgery and also artificial intelligence so i came back i

discussed with my co-workers and we decided to create irkad that means an institute totally

dedicated to research in new technology and training in this new technology since the

the beginning we work on the concept of augmented surgery and if we want really to understand what

is augmented surgery just have to ask the question to a surgeon the first answer is to improve my

surgery i want to improve my eyes i want to see more than what i can see normally and especially

i want to see in transparency so that is the first answer the second answer is i want to do more with

my hand that is augmented and that is robotic surgery and finally i want to have a better

strategy before the operation during and after the operation and that is the role of augmented brain

that means artificial intelligence so when we start by augmented eye what is your motor eye

we have a city scan in 2d here is the example of a collage of carcinoma very difficult to understand

on where is the tumor you see that all the experts say the tumor is on the right side that was during

an advanced course of hepatobiliary surgery and when we have the opportunity to reconstruct in 3d

you are going to see very easy because we can remove the external part of the liver

so we have all the branches we want to remove the biliary tracts the arteries and immediately when

you have just veins you see that the tumor is absolutely not in the right part of the liver

but because we have not the median then it is in the left part and when we apply a virtual clip

you see that we see exactly the vascularization and when we do that on the left part you see that

we remove absolutely the left part of the liver so that is to explain uh how important is the 3d

reconstruction. The same for this amatoma, a young baby, two months old. You see this

big amatoma. It's very difficult to know where the vascularization starts, so we can

navigate inside, and you will see that very easily, we see that we are going to see the

hepatic artery. So we see the hepatic artery here, and we see the two branches going to

to the tumor. So immediately for the pediatric surgeon, it was so easy to start the operation

by this control. And after that, in 45 minutes, the operation was done. Another example is this

double nephroblastome. And you see that with the 3D reconstruction, it is the only way to be

possible to be able to do a conservative surgery, bilateral conservative surgery. And that was what

has been done by the surgical team in paris uh you know that is the first step the first step

is the planning the second step is the simulation so you see here three metastasis of the liver

we know that for the left one it's possible to do a three segmentectomy and for the right one we

want to do an ablation and because we have this possibility of interactivity we can take any

instruments. So we take, for example, the needle here. We are just looking what is the best way

to be in the center of the tumor, and you see that that is a real simulation of the radiofrequency

ablation we plan to do for the patient. We can take also the virtual optic and the virtual camera,

and you see that on the window on the left, we see exactly what we are going to see the day

of the operation so you imagine that with increasing of the power of the

computer it will be more and more realistic to do the simulation of the

operation the day before and certainly one day it will be totally mandatory to

prove that we have simulate the operation before making the real

operation finally after the planning simulation is the fusion of both that is

augmented reality you have the 3d image of the normal image you you lose the fusion of both

images and you have the concept of augmented reality we start this concept by tumors of the

adrenal gland and you see that we publish the results in jama that was nearly 20 years ago

and why because it was possible to reconstruct in trees so this is the first step that is

the planning the planification of the operation we see two main veins and not only one and you

will see that when we dissect the external part of the vena cava we just ask the computer scientists

to do the superior position and we see transparency the exact position of the main veins and when we

start to dissect the renal vein, we see in transparency the polar superior kidney artery

that shows we need absolutely to preserve.

So that is the concept of a formatted reality that we can apply also for some pancreatic

tumors like insulinoma that you see here.

We have the transparency, we know exactly the position of the tumor, the localization,

the landmark with the main duct.

So that is the concept of vision in transparency.

But it is not simple because, you know,

we have a lot of challenge for this kind of augmented reality.

The first one is that we have mobilization of the organ

due to the bracing of the patient.

And that has been solved by the computer scientists

scientists because they have some landmark on the sternum and predictive algorithms concerning

the mobilization of the liver, for example.

The second challenge is the position of the patient, which is not always the same during

the operation and during the CT scan, which has been performed some days before.

And here again, this challenge is solved and they have some algorithm to do the exact fusion

if the patient has a new position.

But the more difficult one is that the first thing we do in surgery, in visceral surgery, in digestive surgery, or urology or thoracic surgery, is to put a retractor, and so we have immediately a new volume, a new localization of the organ.

That is very difficult because we need, the machine needs to understand where is now the final localization of the organ and the volume of the organ.

And to do that, we have created in 2018 another institute just in front of IRGAD, which is the Institute of Image Guided Surgery,

where we tried with our partner siemens to imagine an operating room with all the different

system of imaging technologies that means we have the city scan in the operating room

we have the mri we have the robot size the city scan which is a zero of the pheno we have the

last generation of ultrasonography and that was the end of this construction

just in front of each card and that is a concept that means for complex hepatic surgery we can have

the intraoperative ct scan for arterial phase or venus phase but we know that we need also to see

the biliary tract and for that we need absolutely gmi so we have the mind the operating room

And after that, during the operation, we can see absolutely in 3D with Zico, the different image, and we have a fusion of different images.

So you understand that the steps, the most important one is 3D reconstruction.

For that, we have created a spin-off of IRCAD, which is visible patient, and you see that we can in 3D reconstruct all the different organs.

The problem was that it was long, because for one reconstruction of all the details of the liver, the lung, or the kidney, it was more than between 3 and 4 hours.

But during the COVID, you see, that is really magic because you see that for the lung,

that is applied to lung cancer, in 2019, it was four hours of reconstruction and it was semi-automatic.

And at the end of the COVID, because a visible patient has to reconstruct thousands of cases of lung,

you see that it was nine minutes fully automatic by artificial intelligence using two algorithms

patented in Europe and in the United States. So you see in one year the fantastic power of

artificial intelligence to decrease the time of reconstruction. And now Visible Patient is

an exclusive license with Johnson & Johnson for open laparoscopic surgery and especially

especially for the development of the new robot verb.

Second challenge after the vision is augmented hands,

that is robotics and robotics,

how we can integrate the image in the robot of the future generation.

And here you see the robot Da Vinci.

You see like a GPS in a car,

the 3D modeling which is integrated on an iPad.

And when the surgeon is looking inside the master part of the robot, you see that he has a vision of the camera, but he has also the vision of the 3D reconstruction, which is like a landmark to know exactly the details of the anatomy.

anatomy. The second thing that we have developed in surgery, and I think it's very important for

cancer, for oncologic surgery, is a long-distance surgery to have a team who is able to support

another team. And you remember that in 2021, we described the operation whose name was Lindberg

operation between New York and Strasbourg. That was a total operation, very simple one. It was

a cholecystectomy. But just to prove that today, with the technology, we can do, without any

problem, a total long-distance remote surgery. And that was published in high priority in Nature.

But something is very important is that today, since last year, you have a lot of publications,

publication, Chinese publication, trying to use the 5G for remote surgery, but today it

is only preclinical trials, never more than 3,000 kilometers, with a latency time of 264

millisecond.

And we mentioned that 20 years ago, it was clinical on a patient, it was 14,000 kilometers,

and a time of latency of 155 millisecond,

which was totally unique,

performed by two young engineers of France Telecom.

A world about alimentary tract cancer,

and you know the evolution of what we call

endocrine surgery using flexible endoscopy,

and certainly in the future,

robotics flexible endoscope.

Why?

Because I was absolutely impressed by a conference

of Professor Tanigawa

that was seven years ago.

Seven years ago in Japan,

they have operated more gastric cancer

by endoluminal way

with flexible endoscope

than by conventional

and laparoscopic surgery.

And they say certainly

in the next 15 years,

80% or more of alimentary tract cancer,

if you have early detection,

will be operated by flexible endoscope.

But what we have seen is a lack of triangulation, and you know that to operate, it's difficult to operate with a normal, conventional, flexible endoscope, because we need a triangulation.

So we developed with Carstor-Sondesco, this system whose name was Anubis, and you see that we have a triangulation, we have a mobilization of two instruments, and we can do a lot of manipulation, even a suture, an endo-liminal suture.

but it was difficult to manipulate and it is the reason why we have developed a specific robot

with the team of the University of Strasbourg and it is very interesting here to see how ESD is

possible. On the left it is a South Korean gastroenterologist with an experience of more

than 1000 ESD and on the right it is a surgeon without any experience of ESD but with a robot

And when you compare the image on the left with a gastroenterologist with a huge experience

and on the right with a robot and no experience, immediately you see that on the right it is a real operation.

So certainly the future will be robotic and flexible on the scope.

I want to finish by augmented brain, which is artificial intelligence, just to remember that it is not new.

You know, in all the newspapers since five years, we see a lot of papers concerning artificial intelligence as if it was something totally new.

Now, the first machine learning was elaborated in 1957 by Frank Rosenblatt.

1957, but nobody believed in the project.

And you all know that in 2014, because the machine was a winner against the world champion of the Go game in Korea,

That was the beginning of an explosion of money, support for artificial intelligence.

What we have done is, in the last five years,

elaborate what we call a condor project,

which is a control tower in the operating room.

It didn't exist, but it is possible today,

because we can storage with absolutely low cost all the pretty image,

the external image of the operation, laparoscopic image,

or the physiological system return to work.

So that is an application.

The other one, very interesting,

certainly in the future for cancer surgery,

is how the machine is going to be able

to send an alarm to the surgeon

if the protocol is not the exact protocol.

Here we start with cholecystectomy.

We send hundreds of cases of cholecystectomy

and the machine understands different steps.

And you know that the only way to preserve from a complication like biliary tract, the common biliary tract section, is to have a perfect...