About Caluã de Lacerda Pataca

Olá! I’m a Ph.D. candidate and Fulbright fellow in the Computing and Information Sciences Program at Rochester Institute of Technology (RIT). I am co-advised by Matt Huenerfauth and Roshan Peiris in my research, which deals with speech accessibility and captioning technology. I am also a member of the Center for Accessibility and Inclusion Research Lab (CAIR).

Previously, I obtained a master’s degree in Computer Engineering at the University of Campinas, working with Paula Costa on speech-modulated typography. My thesis can be found here.

I am also a co-founder of Preface, a design studio from Campinas, Brazil. Since 2007, we have worked with clients from many sectors, with a particular focus on educational projects. For my work as a designer there, I have won some awards.

Research interests

My research centers on developing innovative approaches to captions to bridge the gap between how expressive spoken speech is and how the current state of captioning renders text in a dull and ambiguous manner. With a focus on the perspectives of Deaf and Hard-of-Hearing users, I investigate various forms of captions, including affective, prosodic, and haptic, and how they can be used to enhance the expressive quality of captioned content. This research falls within the domains of HCI and Accessibility, and seeks to improve both live and pre-recorded content, such as online meetings and movies, to provide a more inclusive and engaging experience for all viewers.

Publications

TACCESSHassan, S., de Lacerda Pataca, C., Amin, A. A., Nourian, L., Navarro, D., Lee, S., Gordon, A., Watkins, M., Tigwell, G. W., & Huenerfauth, M. (2024) Exploring the Benefits and Applications of Video-Span Selection and Search for Real-Time Support in Sign Language Video Comprehension among ASL Learners In ACM Transactions on Accessible Computing. Association for Computing Machinery (ACM).

CHI’24Honorable mentionde Lacerda Pataca, C., Hassan, S., Tinker, N., Peiris, R. L., & Huenerfauth, M. (2024) Caption Royale: Exploring the Design Space of Affective Captions from the Perspective of Deaf and Hard-of-Hearing Individuals. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems.Presentation

CHI’24Honorable mentionHassan, S., de Lacerda Pataca, C., Nourian, L., Tigwell, G. W., Davis, B., & Wagman, W. Z. S. (2024) Designing and Evaluating an Advanced Dance Video Comprehension Tool with In-situ Move Identification Capabilities. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems.

CHI’23de Lacerda Pataca, C., Watkins, M. P., Peiris, R. L., Lee, S., & Huenerfauth, M. (2023) Visualization of Speech Prosody and Emotion in Captions: Accessibility for Deaf and Hard-of-Hearing Users. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Presentation

IEEE T AFFECT COMPUTde Lacerda Pataca, C., & Costa, P. D. P. (2023) Hidden bawls, whispers, and yelps: can text convey the sound of speech, beyond words. IEEE Transactions on Affective Computing.

Assets’22Best-paper nominationHassan, S., Amin, A. A., de Lacerda Pataca, C., Navarro, D., Gordon, A., Lee, S., & Huenerfauth, M. (2022) Support in the Moment: Benefits and use of video-span selection and search for sign-language video comprehension among ASL learners. ASSETS ’22: The 24th International ACM SIGACCESS Conference on Computers and Accessibility.

de Lacerda Pataca, C. (2021) Speech-modulated typography (Master’s thesis, University of Campinas, 2021). Download

IUI’20de Lacerda Pataca, C., & Costa, P. D. P. (2020). Speech modulated typography: towards an affective representation model. In Proceedings of the 25th International Conference on Intelligent User Interfaces.

de Lacerda Pataca, C., & Costa, P. D. P. (2019). Tipografia modulada pela fala: avaliação de um algoritmo de geração de prosódia visual em textos. In Blucher Design Proceedings. 9° Congresso Internacional de Design da Informação. Editora Blucher. Download

Patents

Costa, P. D. P., de Lacerda Pataca, Caluã. Método para composição de closed-captions com parâmetros tipográficos representando visualmente qualidades acústicas da fala (Pending. Process BR-10/2021-016125-6)

Victorio, R. A. D. S. S., Vargas, G. V., Chufi, F. A., Takahashi, T. Y. I., San Juan, F., de Lacerda Pataca, Caluã, ... & Ribeiro, M. C. M. (2012). Traceability system applied to farm production activities, industrialization and commercialization of bee products. U.S. Patent Application No. 12/596,612.

News

June 24, 2024 Successfully defended my dissertation proposal.

April 16, 2024 Had a great time attending CHI 2024!

January 18, 2024 Two papers we submitted for CHI 2024 were (conditionally) accepted.

November 6, 2023 I had a great time at the Haptics for Symposium, at the University of Borås, in Sweden, where I presented and showcased my ‘Enhancing Affective Captions with Haptic Feedback’ project.

October 23, 2023 I attended my first (in-person) ASSETS!

April 23, 2023 I attended my first CHI!

January 13, 2023 A paper we submitted for CHI 2023 was (conditionally) accepted.

October 7, 2022 I presented a poster at the AI@RIT summit.

August 3, 2022 A paper we submitted for ASSETS 2022 was accepted.

May 19, 2022 My master's thesis, defended in March 2021 but on hold because of a patent application, is now available.

May 11, 2022 Passed my RPA exam.

May 3, 2022 The paper I wrote with Paula Costa, Hidden bawls, whispers, and yelps: can text convey the sound of speech, beyond words?, has been accepted for an upcoming issue of the IEEE Transactions on Affective Computing.

Miscellaneous

May 22, 2023 The University of Campinas Gazette published an article about the research I did during my master's degree. Also included was a fun short Instagram reels preview of the work.

March 6, 2023 Philipp Sokolov published a humerous take on prosodic typography that featured our work.

December 9, 2022 Martin Anderson published an article about our ‘Hidden Bawls’ paper.