Author Archives: imd-lab

Internship at University of Oulu

Hi, my name is Ayaka Matsumoto, M1 student in IMD lab.

University of Oulu

I have been in an internship at University of Oulu in Finland since 7th January. Here, I join the project team that is working on 3D pose estimation from 2D images with Ph.D. student, Lam and under supervision of  Prof. Janne. This project is using Machine Learning, so I’ve learned about it for a month before I started my internship. It is very challenging work for me, because I have less knowledge about Machine Learning. However I hope I will be able to get some results for this project.

By the way, it is basically -15℃ here, and in the morning and night it goes under -20℃! I have never experienced this super cold weather before, but I’ve never experienced this incredible scene where everything is covered with snow as well. It is very beautiful. Since the campus building wall is painted with fresh color, you can enjoy the contrast between white world of snow and colorful world of college.

It is 3pm. It is getting dark.

White tree covered with snow.

Blue & Yellow hallway

Orange & Yellow hallway

Green lecture room


2018 JST-MoST Conference ICT for Accessibility and Support of Older People

Hello everyone! Jayzon (Master’s graduate) here.

Last November 24 – 27, Kato-sensei, Okahashi-sensei (Occupational Therapist from Kyoto University Hospital), Matsumoto-san (M1), Inoue-san (M1), and I went to Tainan city in Taiwan to attend the 2018 JST-MoST Conference on Information and Communications Technology (ICT) for Accessibility and Support of Older People. The conference was organized via a collaboration between different universities from Japan and Taiwan, and was mainly focused on addressing issues related to super-aged societies using ICT.

View of Tainan City from the hotel.

Photo of the conference venue. Photo courtesy of Okahashi-sensei

One of the main events of the conference was the poster presentation. Matsumoto-san presented her current research about developing strategies for promoting and sustaining motivation for rehabilitation, Inoue-san presented about designing, developing, and evaluating an augmented reality (AR) music game for finger rehabilitation, and I presented about integrating the finger pressing evaluation training system (PETS) with AR. (Photos courtesy of Okahashi-sensei)


Towards the end of the conference, 3 best poster awards were presented, and Inoue-san and I received 2 of the 3 best poster awards.


Given that we were already in Taiwan, we did not miss the chance to visit the night market and try the stinky tofu (臭豆腐). Inoue-san and I thought it was delicious, but Matsumoto-san hated it very much XD.

We also had a tour of National Cheng Kung University (国立成功大学) and were introduced to different research laboratories.

Finally, before going back home, we visited the Hayashi department store (林百貨) to shop for souvenirs. Hayashi department store is one of the most iconic department stores in Tainan city, dating back to the period when Taiwan was colonized by Japan. Across the department store is a branch of the Nippon Kangyo Bank (勸業銀行), another iconic building that was also established on the same period.

Hayashi Department Store (林百貨)

Old Branch of Nippon Kangyo Bank (勸業銀行)

Overall, we really enjoyed Tainan city, and we definitely would love to go back. Here’s hoping for a stronger collaboration between Japan and Taiwan in the near future!


Internship at University of Oulu, November 2018

Hyvää päivää!
M2 student Silviya here, been on the second week of my Internship at Oulu University, Finland. First impression ? The weather is super COLD, it’s literally freezing every day. But hey, you can get free coffee in the lab every day ! Yaay coffee !! ❤

I’m assigned in the Faculty of Information Technology and Electrical Engineering, laboratory of Center for Machine Vision and Signal Analysis (CMVS), under the supervision of Professor Janne Heikkilä. My first day was amazing, of course.. as always I got lost a bit when I tried to find the lab, but the good point I was able took my own time to explore the university, went through most Faculty before I got to the right lab (Yaay again!). Here, in CMVS lab. their research covers broad and various fields, including Neural Network, Augmented Reality, and 3D reconstruction.

Once you get inside the laboratory area, you will see a screen display, with Face Analysis demo running in real time to do gender recognition and also facial expression recognition. But then “Female = 0 “. Finally, I’m an alien and can’t be detected ! (Yaay again??)

I’m undetected. Yaay !

Continue reading


Greetings Everyone!
Damien Rompapas, D3 student, Reporting for Duty!

I participated in two conferences back-to-back:
ACM UIST 18 (31st symposium on User Interface Software and Technology) in Berlin, Germany
IEEE ISMAR 18 (17th International Symposium on Mixed and Augmented Reality) in Munich, Germany

Where I presented a game developed as part of my research work as a demonstration.

I presented HoloRoyale: A Large Scale High Fidelity Augmented Reality game
(you can see a video explaining the work here: video )

The presentation was quite enjoyable, I gave a demo and teaser many people wouldn’t forget

Commander Rompapas ready to present the 1 minute teaser!

Of course I enjoyed various other activities, such as enjoying the local food in Munich, one restaurant was quite special, you could order a specific cut of meat and they cooked it up for you (which leaves a bone one can have a little fun with!)

Goofing off at a specialist steak restaurant

MIRU 2018

Hello everyone.
I’m Ryo Akiyama, D2 student.

I participated in MIRU 2018 @Sapporo on August 5th to 8th. MIRU (Meeting on Image Recognition and Understanding) is one of the largest conference about computer vision in Japan.






I had poster presentation this time. The title is “Expanding perceptual color range of a projector by inducing color constancy (色恒常性を利用したプロジェクタの色域の知覚的拡張).” In MIRU, all the presenters (even invited speakers) need to have poster presentation. In other words, we can meet all the presenters at the poster presentation room. I don’t know this is the reason, but many participants were discussing with presenters at poster room and they look very active.

Of course, I also discussed with many people about my research. And I got “Interactive Presentation Award.” This is a kind of “best poster award”, and 7 presentation (from 224!!) got this award. I am very glad to have this award, and I feel thank to my supervisors who support me a lot.

I conclude this post with some photos.
Thank you for reading.


poster presentation









when I got the award

people in Hokkaido eat a parfait after drinking


Hello, I’m Haruka Matsumura, in our laboratory’s event organizer committee.
In early May, we held Welcome Party for new M1 students who entranced in this year April.
(My post was so delayed X( )

In this time, we have 6 new Japanese students joining our lab, and also 2 research students in our lab becoming new M1.
So, we have 8 new M1 students now.
We’re happy coming many new students!

Group Photo

Visit to Mississippi State University

Hello everyone!

This is Jayzon Ty, an M2 student, and welcome to my first blog post!

Last February 12 to March 16. I was at the Center for Advanced Vehicular Systems (CAVS) in Mississippi State University (MSU).

Center for Advanced Vehicular Systems Building

While there, I was working under the lab of Dr. J. Edward Swan II, who specializes in human factors of virtual and augmented reality, particularly in depth and layout perception. The lab is relatively small compared to the IMD lab, in terms of size of room and number of people, but this creates a very warm atmosphere where everyone is really close with each other.

Inside Dr. Swan’s lab

Photo with some members of the lab. Unfortunately, I did not have a proper photo with everyone in the lab itself, so here’s a photo of us eating two giant pizzas.

During my stay, I helped Dr. Swan with the SharpView project. In a nutshell, SharpView is a method for counteracting image blurring when viewing an image out of focus, i.e. “sharpening” it. I worked on implementing features that Dr. Swan wants to add to SharpView, as well as investigating other approaches for the “sharpening” part. In the process, I got to work with the haploscope, an AR display that can display virtual objects in different focal distances by changing the lenses in the system. I also worked with the SMI Eye tracker, a very expensive eye-tracker that is worn like eyeglasses (which I don’t have a picture of, unfortunately).

The haploscope, an AR display capable of displaying virtual contents at different focal distances.

As they say, everything in the US is big, and Mississippi State University is no exception. The campus is so big that it has its own lake, and as well as a lot of sports stadiums. In order to get to the lab from my dormitory, I can take a bus, or I can take a 30-minute walk along the scenic route. I almost always choose the latter.

Walking path to the lab

In-campus lake

Basketball Stadium

Butler Hall, Computer Science Department

I also had the opportunity to attend a women’s basketball match, thanks to Dr. Swan. Apparently, the women’s team was undefeated at the time, and *spoilers*, they retained their undefeated streak in this match.

Mississippi State University versus Texas A&M

The team won the match, maintaining their 28-win streak for the season!

Speaking of big, here is a photo of a burger that I ate one time. Note the size of the onion rings with respect to the burger.

Bulldog Burger with Onion Rings

Unfortunately, public transportation was a bit lacking, so I was not able to go to a lot of places outside of campus. All in all, I could say that I managed to go to three major places outside of campus.

The first one is the town of Starkville, where MSU is located. This area in particular is around a 15 minute drive from campus. The town has a very chill atmosphere, which I really liked. We often go here for lunch, and sometimes for dinner as well.


Fun fact: Dr. Swan is a computer scientist by day, and a musician by night, so he goes to a lot of places to perform with his band, the “Shane Tubbs Band”, as their bassist (I really want to learn the bass sometime). So, for my second major trip, we went to the town of Westpoint, where Dr. Swan and his band performed in a music festival, which served as a fundraiser event for the treatment of a person with cancer. Funnily enough, the event venue is in a farm in the middle of nowhere.

Dr. Swan and another kid playing with an anthill. Poor ants.

Dr. Swan (while talking to the kid): Do you know what cows are made of? Hamburgers!

Dr. Swan performing with his band at the music festival. Unfortunately, I wasn’t able to take a closer photo.

The last major trip was a trip to another state, Atlanta, which was a 4 hour drive from the campus. One of the students in the lab needed to get a German visa in order to attend IEEE VR 2018, so I went with him to Atlanta, where the German embassy was located. We left campus at 3am, and arrived at around 8am (+1 hour difference).

As we were heading home, we also passed by the University of Alabama. Same as MSU, the campus was also very big.

Football stadium in University of Alabama

Unfortunately, all good things must come to an end, so before I went back to Japan, we had a farewell dinner at a restaurant in Starkville. The food was so big that I wasn’t able to finish it, and I had to get a take out box so I could eat it for lunch the next day.

The picture makes it look small, but the chicken is actually really big, probably around 3/4 the size of an HTC Vive.

Farewell dinner with Dr. Swan and some of his students. They flew for IEEE VR 2018 the next day.

On the way back to Japan, I had a layover in Incheon International Airport in Seoul. I happened to find this demo inside the airport: VR Flying. Unfortunately, I was already too tired at this point, so I did not give it a try. Maybe we should have a similar setup in the lab?

VR Flying demo inside Incheon International Airport

Overall, I really enjoyed working with Dr. Swan and his students, as well as the experience of being in the US. I really learned a lot in this experience. This trip also introduced a lot of first’s for me, e.g. first trip to the US, first 15~ hour flight, first jet lag experience, first trip to South Korea (even though it’s only a layover), and more. If you ever get the chance to go to MSU and work with Dr. Swan, I highly recommend going for it!

Finally, I would like to thank the Interactive Media Design laboratory, especially Dr. Alexander Plopski who organized this trip, and Dr. Edward Swan for making this experience possible. I would also like to thank Dr. Swan’s students, as well as his wife, Mrs. Courtney Swan, for making my stay in Mississippi a very enjoyable and memorable one. Until we meet again!

IEEE VR 2018 (Akiyama)

Hello, I’m Ryo Akiyama, D2 student.
I attended to IEEE VR 2018 in Reutlingen, Germany (March 18th ~ 22nd).
It was very cold, and snow still remained in the city.

Welcome to IEEE VR 2018 !


Stadthalle Reutlingen

From IMD lab, Oral, Nicko, and I attended this year. Oral and I had poster presentation, and Nicko had conference paper presentation. Nicko presented his work about comparing efficiency of AR and VR environments for memorizing spatial position. Oral presented his work about visualizing knee movement for supporting bicycle training with less risk of injury.

Nicko’s presentation


Oral’s fast forward presentation

I presented about light projection technology to control object color by inducing visual illusion. Projectors only can add colors to real objects by overlaying light projection. However, when the object is not white, only a limited colors can be presented by projection. Our project want to overcome this limitation by inducing one of the visual illusion, color constancy. I presented current results with my poster. Many people more than I expected came to my poster. I had many discussion with them. It was great time for me to know many opinions or questions.

The largest number of participants ever!

IEEE VR 2019 in Osaka, Japan.

Next year, IEEE VR 2019 will be held in Osaka, Japan. Nowadays, AR and VR have became very popular in many field. I expect many participants more than this year come to IEEE VR in Osaka.

IEEE VR 2018

My name is Nicko Caluya, and welcome to my first blog entry here. I am an M2 student.

Last week (March 18-22), I attended IEEE VR 2018 with labmates and professors at Stadthalle Reutlingen, at the southern part of Germany.

Stadthalle at night, all bright and colorful

Since we attended the full conference, we sat through two days of workshops, and three days of the main conference. I would like to point out some highlights from each day.

Prof. Swan’s Presentation

On the first day, I attended a tutorial session by Professor J. Edward Swan II regarding the statistics and the replication crisis. In the afternoon, I filled in the latter half of the day with a displays tutorial, where speakers talked about High Dynamic Range (HDR) and gaze-aware displays. I hopped my way through second workshop day, going from the calibration tutorial in the morning, to a tutorial in Web 3D in the early afternoon, to the last session on a workshop on perceptual and cognitive issues in AR.

Oral trying out the demo at the displays tutorial

Aside from attending the conferences, we had the chance to visit two institutes. On the second workshop day, we rode the bus to the Fraunhofer Institute at Stuttgart. The researchers took us to the Hazel Hen (a high performance computing system with peak performance of 7.42 PFlops), to an awesome CAVE (cave automatic virtual environment), to an actual (stationary) car for a driving simulator, among many awesome demonstrations. I took a picture of us wearing the glasses in the CAVE, and I can say we can definitely form a cool boy band!

The new boy band from IMD

The day after, we toured the Max Planck Institute at Tübingen to see giant rooms housing a VR CyberMotion Simulator and a cable robot where you can be suspended inside an icosahedron.

I was with Oral and Akiyama, who delivered their poster presentations on the first and second days of the main conference, respectively. Many professors and students from different universities in Japan also attended the conference. Immediately, I was already excited at the prospect of meeting people behind the cited works from our lab meetings or lessons. These names will finally have faces and characters!

Akiyama explaining his projection work 

While this was not my first time for attending a conference, I was still amazed at all the new wonderful things I have learned from the presentations. Veterans from both the academe and the industry presented their keynote presentations as if they have rehearsed all their lives for their own moments. What made me entertained so much was this keynote Dr.-Ing. Oliver Riedel, presenting the 25 years of VR in industry as an Einstein look-alike.

Dr.-Ing. Riedel’s presentation

For me, the term “virtual reality” is already such a huge domain, and I believe that the conference made me realize just how much bigger it is. For example, results from “Avatars and Virtual Humans” session used statistical treatments for within- and between-subjects studies, primarily because they deal with human perception. In another, the papers from the rendering session offer various computer graphics techniques and hardware configurations to make specific VR experiences better. And of course, the multimodality session showcased papers which featured the VR of the other senses (sound, smell, touch) which often gets overlooked by the bias of VR research towards sight. Many paper presentations amused me across all sessions. Some presentations had slides designed very elegantly. Some carried me away with either the storytelling or the amount of data they were able to analyze on one study alone.

On the final day, I presented first on a session about “Selection and Pointing”. I was surprised to hear someone say “That’s our paper!” when I was scrolling through my slides to check if the presentation worked correctly. I presented our work on comparing AR and VR environments, to see which of the two can efficiently transfer the skill of memorizing locations of objects from training to the actual scenario. During the presentation itself, I was quite nervous, but not because of the size of the screen behind me or the big hall aptly named “Grosser Saal”. I was nervous to go over the time limit and not say what I need to say. So, it was a really nice feeling to finish the presentation on time, with enough time to answer two short questions. In the afternoon, I was even able to try out a demo by Daimler, riding on a Mercedes Benz passenger seat in the snowy streets of Reutlingen.

My presentation. I am so tiny compared to the screen!

As a person who cherishes the first times, it was definitely a great experience to showcase my first research work as a first author, on my first big conference as a graduate student, on my first time in a European country (the farthest I have traveled so far – 17 hours, 9000+ kilometers!). It felt like the whole process of getting there – from conducting the experiment, to writing and violently revising the paper, to preparing all the necessary paperwork to enter Germany and attend the conference – was such a humbling experience. I would like to take this opportunity to thank everyone who made it possible for me to go through all this without fainting. Thank you most especially to Alex-sensei and Jayzon for sticking out with me in order to get the paper done. Thank you to IMD faculty and students who shared valuable comments as I showed my progress and rehearsed my presentation before leaving.

Oh, and Germany was nice, except that the weather did not cooperate (sunny in the morning, snowy in the afternoon). I had so much fun… drinking lots of beer and eating lots of bread (and schnitzel, and kebabs, and currywurst… the list goes on).

Turkish food are popular in Germany; they have a shop at almost every street corner

Tübinger Tor (the structure you see in IEEE VR 2018 logo) on a snowy afternoon

Currywurst and fries at Frankfurt Station before leaving

From now on, I am more determined to continue with my research and publish better works, with more drive and with a broader source of ideas. As I have found out from the closing ceremony, IEEE VR will be in Osaka. I hope I can participate and present again.

For now, stay tuned. To be continued. 「つづく」

IEEE VR 2018

Hi. D2 Oral is here.

From March 17th to 22nd, we visited Reutlingen in Germany to participate IEEE VR 2018. Me and Akiyama had a poster presentation and Nicko introduced his work in the form of a conference paper.  Overall I believe it was a valuable experience for all of us to learn about new technologies and create connections with researchers from all around the world.

The conference took place in Stadthalle Reutlingen for a total of five days; two for workshops and tutorials, three for the actual conference. Although the name of the conference is VR, it included research focusing on MR, AR, haptics or distinct interaction technologies. I was mostly interested in “Tutorial on Cutting-Edge VR/AR Display Technologies” due to its focus on “Motion-Aware Displays”.

As always, I will cut it short and let pictures do the talking about this episode of my studies. Detailed info on conference and awards given will probably be shared by my colleagues anyways… 😉


Stadthalle Reutlingen

You can always learn new laws

Spätzle and Weissbier

You can always listen an acquaintance’s presentation

Super computer at Max Planck Institute

Somebody is doing cycling research?

A nice work on tactile cues

Nicko’s presentation