On June 9th I had the pleasure of “Zoom” visiting the Gratton Lab at the Beckman Institute, University of Illinois, kindly invited by my great collaborator and ex-Aaltonian Ana Triana Hoyos. While we could easily have talked about fMRI functional connectivity (a topic very close to all our hearts), my more recent passion for research ethics, research integrity, and AI in research workflows ended up taking our 1.5h together with Caterina and all the lab members discussing how to use (and how not to use) AI tools in research. The slides are available here (PDF). Below is the full Q&A that the lab had prepared in advance, with my answers and reflections.
Q&A
How do you classify the type of data that we use? Do you have specific AI use cases for each type of data?
Organisations usually define 3 or 4 levels of “classification of information”. A typical classification is
- Public: anything that can be shared with anyone, e.g. wikipedia pages
- Internal: meeting notes, expenses, etc something not for the general public but it does not require confidentiality (e.g. internal university pages)
- Confidential: something that only a handful of individuals should have access to (e.g. participants’ brain scans)
- Secret: something that only a handful of individuals should have access to + in case of data breach there is serious harm caused
If the work I am doing is work that I could easily do in public (e.g. using a github repo that is open or doing that work during some zoom teaching session), then any AI is good. I have to remember that the more I disclose to the AI system, the more is the likelihood that I get scooped or plagiarised. In the moment that I have confidential data, then I need a remote endpoint that ensures a level of security/privacy OR a local endpoint that can be fully controlled. Some tools should always be treated as if you are typing things in public: my personal list of tools that interacting with them is like doing it publicly: Meta AI, Grok (xAI), Deepseek.
Where are you using AI comfortably, and where not? (Enrico’s own AI disclosure)
My rule of thumb is a 2x2 matrix: expertise X risk: if what I delegate to the AI is a task that I am good at, then I know that it is fast and reliable for me to verify its accuracy. If I delegate to the AI a task that I do not have the competence to review (e.g. an overview of the literature on a topic I know almost nothing about), then I just treat it as a tool to brainstorm with, getting me started with suggestions and then move to read real sources.
| Low expertise (I know very little about the topic) | High expertise (I know a lot about the topic) | |
|---|---|---|
| High risk (e.g. something that will be submitted for peer review: text, analysis code, figures, tables, references) | Use AI only to brainstorm about what real sources one should read to know more about the topic or to get help with keywords to find those sources using standard tools like scopus, scholar. My use cases: asking and learning about statistical methods that I will use later. | Use AI to delegate simple tasks that you are an expert on (e.g. software coding, or rephrasing a sentence you wrote). Review absolutely everything as if you had to re-type it manually. This is where overconfidence bites researchers most. My use cases: simple code snippets, function by function, copy pasted and run manually by me. Asking AI to suggest improvements for text, AI marks changes in bold and then I manually apply some of the suggestions to my own text. |
| Low risk (e.g. a webpage for an event, a text for a social media post) | Use AI, but accept that there is a non-zero chance of slop. My use cases: CSS for web pages, vibe coding of simple games for children, coding small local web apps (e.g. local translation of documents) | Use AI to delegate most of the work. Check that the output makes sense. My use cases: documentation for script and software, summaries of longer texts I wrote or I am familiar with, outline of teaching materials based from my recordings of teaching. |
Note 1: The level of sensitivity of the data will shift the risks higher.
Note 2: There is one case that falls between multiple sectors of the matrix: when the verification part can be formalised through testing (e.g. automated testing of code with unit tests). Then if the automated tests are well defined, the verification step is done by the automation. Again the level risk might change how much we trust or need to control the automated tests.
Compliance: how do you get people to abide by AI guidelines? Are there any incentives that are useful?
I personally rely on Trust rather than Fear, I teach this topic at the research ethics course for all doctoral students and the main message is to just be honest and transparent. Because I don’t trust AI in general, I cannot trust AI detectors. Turnitin AI detector has caused harm to students, and even if we don’t take its result seriously, it will anyway bias the person who is using it to evaluate the work produced by others. AI guidelines in the end are very simple: “do no slop, and declare AI use” but people are lazy and think they can cut corners and not say that they did cut corners: Trust cannot be ensured, so Fear comes into play: I am glad that journals are taking fabrications seriously, with now 1 year ban by some publishers if AI fabrications are submitted (e.g. fabricated references).
How much effort should we put into memorising little details (e.g. a hidden Matplotlib feature)?
In this case: It is more important to memorise where the manual page is (and how to read it), than the actual feature. One day AI won’t be there, and we will need to read the manual. In general, it is the development of critical thinking. The experts of the futures are not those who manage to use AI agents for anything in their life today, they are those who are still reading books slowly and keeping their thinking sharp: they are building resilience for when AI does not work perfectly, just like the 7 year old is learning the multiplication tables without calculator.
Do you think people are being more productive with AI? Or less?
There are conflicting findings in the literature (more ref later). If we consider one domain where AI is very good at (coding) it seems people are more productive. But the cost is that more unmaintainable code is produced, and prices of AI for coding are skyrocketing. In research: productivity can mean more papers, but peer review is what it is. I personally think that AI should never be a peer reviewer, so even if some bottlenecks where removed from researchers, they come back as limits of the peer review system.
How do you identify AI when it is used problematically in other work?
As I wrote above, AI detection tools are useless unfortunately. If I see how others are using AI, I can remind them of the risks, but often we only see final products that look perfect without slop. I hope that Trust is the fundamental value at the core of all interaction between researchers and the future of science.
In a review process, how to spot for fabrication of references?
Good old manual searching is the only way as far as I know. In this paper they provided a workflow for doing that:
- If doi / pubmedID available, check if it resolves to the same cited article.
- If mismatch, then flag the reference
- Check that it was not a false positive (typo, abbreviations)
- Search for title in 4 databases (Pubmed, Crossref, openAlex, Scholar)
In general it is a tedious job for reviewers, it would be great if journals would spot those at submission rather than doing peer review.
Are journals/foundations using AI for reviewing papers/grants? How are they planning to use it?
At least Horizon Europe has banned the use of AI for reviewing grants. In the conference ICML they did an interesting experiment, I can expand on it if you want.
How are admins monitoring AI-agents use in HPCs?
This is another one of those difficult situations: we don’t want to ban everything, because that will just send people away and in the end the risks for themselves and their research will be even bigger, but we also don’t want to let agents take over all computing nodes of the cluster. We have set up a dedicated login node for those users where they can run their agents. We made a long list of “what could possibly go wrong with AI agents on HPC?”. However, the risk of agents is that 1) the more they are used, the more they are trusted, user start accepting any suggestion 2) they can pull anything from the internet, a malicious piece of text in a readme file of a git repository is enough to cause real damages or data breaches.
AI for review process: do you use agents (even local) to review parts of the journal review process?
Absolutely not, not even to fix my typos and italo-english-grammar.
How to disclose AI use in submitted manuscripts?
We don’t have a standard template at Aalto, Elsevier recommends to have a section. Here an example:
Declaration of generative AI and AI-assisted technologies in the manuscript preparation process
During the preparation of this work, the author(s) used [NAME OF TOOL / SERVICE] in order to [REASON]. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.