jodischneider.com/blog

My Information Quality Lab presents 14 posters as part of the iSchool Research Showcase 2023, Wednesday noon to 4:30 PM in the Illini Union. View posters from 12 to 1; during the break between presentation sessions 2:15-2:45; and 3:30-4:30 PM.

Visualizing Race in Medicine
Chris Wiley

Three-Dimensional Archiving of Native American Artifacts at the Spurlock Museum
David Eby

Harold Baron Digital Archival Research and Publication Project
Divya Pathak

Disinformation Tactics and Designing to Deceive
Emily Wegrzyn

Who Needs a Main Entry, Anyway?
Liliana Giusti Serra, José Fernando Modesto da Silva

Epistemological Responsibility in Law and Science: Sharing the burden
Ted Ledford

How Computable is Scientific Knowledge?
Yuanxi Fu

Unified Framework for Evaluating Confidence in Research Synthesis
Hannah Smith, Yuanxi Fu, Jodi Schneider

Using argument graphs to audit reasoning in empirical scientific publications
Heng Zheng, Yuanxi Fu, Jodi Schneider

Activist Organizations and Their Strategies to Influence the Legalization of Medical Cannabis in Brazil
Janaynne Carvalho do Amaral, Jodi Schneider

Assessing Citation Integrity in Biomedical Publications: Annotation and NLP Models
Janina Sarol, Shufan Ming, Jodi Schneider, Halil Kilicoglu

Can ChatGPT Augment PDF-to-Text Conversion Errors in Scientific Publications?
Janina Sarol, Xuhan Zhang, Tanisha Roman, Jodi Schneider

Analyzing Retraction Indexing Quality in Subject-Specific and Multidisciplinary Databases
Malik Salami, Jodi Schneider

How Knowledge Intermediaries Gather and Make Sense of COVID-19 Information: An Interview Study
Togzhan Seilkhanova, Jodi Schneider

[Updated: 14!]

The Washington Post quoted NIH researcher Paul Hwang: “Amazing findings in medicine are sometimes based on one patient”.

The findings here are a breakthrough discovery in a disease called ME/CFS – commonly known as chronic fatigue syndrome or myalgic encephalomyelitis – which led to a recent PNAS paper. This is an amazing moment: Without biomarkers, it’s been a contested disease “you have to fight to get”.

What really strikes me, though, is the individual interactions that created a space for knowledge production: an email from one citizen (Amanda Twinam) to one scientist (Paul Hwang); “serendipitous correspondence” from another scientist (Brian Walitt) with access to “an entire population” (9 of the 14 tested for the PNAS paper were similar to Amanda). Reading the literature, writing well-timed correspondence, and “hearing about” synergistic work going on in another lab all seem to have contributed.

Mady Hornig, a researcher not involved in the project, told the reporter: “It’s not very common that we do all of these … steps, having doctors who are really persistent about what is happening with one individual and applying a scientific lens.”

But what if we did?

Dumit, Joseph (2006). Illnesses you have to fight to get: Facts as forces in uncertain, emergent illnesses. Social Science & Medicine, 62(3), 577–590. https://doi.org/10.1016/j.socscimed.2005.06.018

Wang, Ping-yuan, Ma, Jin, Kim, Young-Chae, Son, Annie Y., Syed, Abu Mohammad, Liu, Chengyu, Mori, Mateus P., Huffstutler, Rebecca D., Stolinski, JoEllyn L., Talagala, S. Lalith, Kang, Ju-Gyeong, Walitt, Brian T., Nath, Avindra, & Hwang, Paul M. (2023). WASF3 disrupts mitochondrial respiration and may mediate exercise intolerance in myalgic encephalomyelitis/chronic fatigue syndrome. Proceedings of the National Academy of Sciences, 120(34), e2302738120. https://doi.org/10.1073/pnas.2302738120

Vastag, Brian (2023, September 19). She wrote to a scientist about her fatigue. It inspired a breakthrough. Washington Post. https://www.washingtonpost.com/health/2023/09/17/fatigue-cfs-longcovid-mitochondria/ Temporarily open to read via this gift link.

The ScienceNLP Lab and the Information Quality Lab at the School of Information Sciences (iSchool) are seeking a University of Illinois at Urbana-Champaign student to read and annotate health-related literature. Knowledge and training and knowledge in a medical/life sciences field such as biology, chemistry, bioinformatics, food science and nutrition, or bioengineering is essential. The hourly will work an average of 10 hours per week for spring semester, under the co-supervision of Dr. Halil Kilicoglu and Dr. Jodi Schneider. The project focuses on assessing biomedical publications for citation accuracy and integrity. Your role in this project will be to locate citation statements in biomedical articles and assess their accuracy with respect to the cited articles. You will collaborate with other annotators on this task. This work is part of the project Natural Language Processing to Assess and Improve Citation Integrity in Biomedical Publications, funded by the Office of Research Integrity (ORI).

Project Description: While citations play a fundamental role in scientific knowledge diffusion and research assessment, they are often inaccurate (e.g., citation of non-existent findings), undermining the integrity of scientific literature and distorting the perception of available evidence. A recent meta-analysis showed that 25.4% of medical articles contained a citation error. A bibliometric analysis revealed that inaccurate citations of a letter published in 1980 may have contributed to the opioid crisis. The project will develop and validate resources and models that aid stakeholders in assessing biomedical publications for citation accuracy and integrity. The publicly available annotated corpus you help create will be used to develop natural language processing/artificial intelligence (NLP/AI) models for assessing reporting quality in biomedical articles.

Duties include:

• Reading and annotating biomedical publications for citation integrity/accuracy
• Contribution to development of annotation guidelines
• Contribution to scientific presentations and publications

Required qualifications:

• Background in a field such as: medicine, life sciences, including biology, chemistry, bioinformatics, food science and nutrition, bioengineering, or a related field.
• Excellent English reading comprehension skills
• Excellent communications skills in written and spoken English
• Excellent analytical/critical thinking skills
• Effective time management skills, attention to detail

Preferred qualifications:

• Interest in topics such as trustworthy science, research rigor/quality, reproducibility
• Interest in biomedical data science, bioinformatics, or related fields
• Availability for multiple semesters

Interested candidates should send their CV/resume and a short statement of purpose drawing attention to their training in medicine or life sciences (e.g., biology, chemistry, bioinformatics, food science and nutrition, bioengineering, or a related field) to Halil Kilicoglu (halil@illinois.edu) and Jodi Schneider (jodi@illinois.edu). Review of applications will begin immediately. Applications will be accepted until the position is filled.

Posted on Handshake and the Virtual Job Board.

For diagnosing problems with time management, the best thing I’ve found so far is Francis Wade’s idea of “skill ladders”, described in the 2014 Perfect Time-Based Productivity: How to rescue your peace of mind as time demands increase (Amazon; book website).

For instance, the “capturing” ladder has 7 levels, with the bottom “not even trying to commit tasks to memory” and the top “always using a backed up electronic device that is never far away”:

He lists multiple ladders:

• Capturing
• Emptying
• Tossing
• Acting Now
• Storing
• Scheduling
• Listing
• Switching
• Interrupting
• Warning
• Reviewing

A downloadable PDF form provides a checklist to identify where you are on each ladder.

A summary of his diagnosis-based approach to managing time demands is available from The Evergreen Guide to Choosing Your Next Task Management App (2017).

More recently he organized the Task Management & Time Blocking Virtual summit (2020 & 2021 & 2022, with the 2023 event planned for Thursday through Saturday March 2-4). You can find more of Francis Wade’s recent thinking via Framework Consulting and in his Quora answers.

Start Date: ASAP

Descriptions, Responsibilities, and Qualifications
This project offers an excellent opportunity for a University of Illinois Urbana-Champaign MSLIS student interested in metadata, data quality, database search, information retrieval and related topics. The incumbent will collect information about how well databases track retracted information, under the mentorship of Dr. Jodi Schneider, Assistant Professor and Director of the Information Quality Lab. The project will produce data analyses and reports to support a NISO Working Group in information gathering about how to improve metadata quality and display standards for retracted publications, in the Alfred P. Sloan foundation grant “Reducing the Inadvertent Spread of Retracted Science II: Research and Development towards the Communication of Retractions, Removals, and Expressions of Concern Recommended Practice”.

We will first search multidisciplinary databases (Scopus and Web of Science) as well as other sources (e.g., Crossref, Retraction Watch) for retracted publications. Then, we will compile a list of known retracted publications across these sources. We will compare across sources to identify retracted publications that have inconsistent information about whether or not they are retracted. We will also calculate which percentage of retracted publications indexed in the source are correctly indexed as retracted. We will then investigate how retractions are indexed in specific domain databases, using established retraction type indexing in biomedicine (PubMed, PubMed Europe)and psychology (PsycINFO), and investigating how retraced publications are indexed in chemistry (CAS SciFinder) and engineering (IEEE Xplore). We will also manually check indexing on a small dataset in search engines such as Google Scholar and Semantic Scholar.

Duties include:

• Searching databases
• Collating publication data
• Deduplicating publication data
• Documenting all aspects of the projects
• Producing project memos and reports

Required Qualification:

• Enrollment in the Master’s in Library and Information Science program at the University of Illinois at Urbana-Champaign
• Interest in topics such as metadata, data quality, database search, etc.
• Interest in quantitative research using publications as data
• Detail orientation
• Excellent communication skills in written and spoken English

Preferred Qualifications:

• Available for continued work in spring 2023
• Project management experience
• Experience with quantitative data
• Experience in database searching
• Experience manipulating data using spreadsheet software (e.g., Excel) and/or scripting languages (e.g., R or Python)
• Interest in reproducibility and open science
• Interest or experience in writing research reports and/or publications

Compensation: paid as a graduate hourly through the University of Illinois, $20/hour for 10-15 hours a week.

Application Procedures: Interested candidates should send a cover letter and resume to Dr. Jodi Schneider in a single PDF file named Lastname-metadata-hourly.pdf to jodi@illinois.edu

Review of applications will begin immediately. Applications will be accepted until the position is filled. All applications received by Sunday October 2, 2022, will receive full consideration.

Posted on Handshake and on the iSchool website

Applications will be reviewed on a rolling basis each Monday until the position is filled, with first review on Tuesday July 5, 2022.

The School of Information Sciences (iSchool) at the University of Illinois at Urbana-Champaign seeks a Postdoctoral Research Associate, mentored by Dr. Jodi Schneider, to contribute to research projects in the Information Quality Lab in the areas of scientometrics, argumentation, and scholarly communication.

Work arrangements

• Remote work within the United States or on-site/hybrid in Champaign, Illinois.
• Two-year appointment with the possibility of renewal.
• Salary commensurate with experience. University health insurance benefits.
• Professional development funding for research-related expenses.

Information Quality Lab: Expected Work
As part of the iSchool’s Postdoctoral Research Associate Program, the selected applicant will receive mentoring and community support to prepare for permanent appointments both inside and outside of academia. The selected applicant will work with Dr. Schneider to create an individual development plan.
The selected applicant will lead, conduct, and publish research, in an interdisciplinary information science research environment. Research contributes to two sponsored projects:

Scientometrics supporting metadata/display standards development in scholarly publishing

• To what extent is authoritative information on retracted papers consistently available and accessible in a variety of field-specific and multidisciplinary databases?
• How does retraction of code and datasets impact related publications and what are the legal, social, and ethical ramifications?
• What are the implications of specific instances of successes and failures in the metadata pipeline, for designing consensus practices?

Document analysis to understand how journalists, activists, Wikipedia editors, and other knowledge brokers assess info on COVID-19, climate change, and artificial intelligence & labor

• Construct corpora of news, social media posts, etc. quoting scientific products
• Lead argumentation and framing-focused document analysis
• In collaboration with the PI and PhD student Research Assistant:
  • Create an information behavior model combining interview and document analysis
  • Collaborate with public libraries to develop a toolkit of services for public libraries
  • Design a scale-up project

Required Qualifications

• A PhD in any field (including, but not limited to, informatics, information sciences, library & information science, digital humanities, or computational social sciences, e.g., communications, anthropology, etc.).
• Research interest in one or more of the following: scientometrics, document analysis, altmetrics, argumentation analysis, science of science, scholarly communication, academic publishing, public policy, public understanding of science, argumentation mining, and text mining.
• Interest in interdisciplinary research.
• Excellent critical thinking, written and spoken English, and project management skills.

Preferred Qualifications:

• Evidence of interdisciplinary research through scholarly publications or translational / implementation science projects.
• Interest or experience in data-intensive and/or mixed methods research.
• Publications in areas related to the research.
• Interest or experience mentoring undergraduate and graduate student research.
• Experience in working with and interested in working with a diverse group of students, faculty, and staff and the ability to contribute to an inclusive climate.

Application Materials – send by email to Dr. Jodi Schneider jodi@illinois.edu:

• Current CV
• Short statement of interest

Questions about the position can also be sent to Dr. Jodi Schneider at jodi@illinois.edu.

Official ad on the iSchool website

The ScienceNLP Lab and the Information Quality Lab at the School of Information Sciences (iSchool) are seeking a University of Illinois at Urbana-Champaign graduate student to read and annotate health-related literature. The graduate hourly will work an average of 10 hours per week for up to 9 months, under the co-supervision of Dr. Halil Kilicoglu and Dr. Jodi Schneider. The project focuses on assessing biomedical publications for citation accuracy and integrity. Your role in this project will be to locate citation statements in biomedical articles and assess their accuracy with respect to the cited articles. You will collaborate with other annotators on this task. This work is part of the project Natural Language Processing to Assess and Improve Citation Integrity in Biomedical Publications, funded by the Office of Research Integrity (ORI).

Project Description: While citations play a fundamental role in scientific knowledge diffusion and research assessment, they are often inaccurate (e.g., citation of non-existent findings), undermining the integrity of scientific literature and distorting the perception of available evidence. A recent meta-analysis showed that 25.4% of medical articles contained a citation error. A bibliometric analysis revealed that inaccurate citations of a letter published in 1980 may have contributed to the opioid crisis. The project will develop and validate resources and models that aid stakeholders in assessing biomedical publications for citation accuracy and integrity. The publicly available annotated corpus you help create will be used to develop natural language processing/artificial intelligence (NLP/AI) models for assessing reporting quality in biomedical articles.

Duties include:

• Reading and annotating biomedical publications for citation integrity/accuracy
• Contribution to development of annotation guidelines
• Contribution to scientific presentations and publications

Required qualifications:

• Excellent English reading comprehension skills
• Excellent communications skills in written and spoken English
• Excellent analytical/critical thinking skills
• Effective time management skills, attention to detail

Preferred qualifications:

• Background in a field such as:
  • life sciences/medicine, including biology, chemistry, bioinformatics, foodscience/nutrition, bioengineering, or a related field
  • library/informationsciences
  • linguistics
• Interest in topics such as trustworthy science, research rigor/quality, reproducibility
• Interest in biomedical data science, bioinformatics, or related fields
• Availability for multiple semesters

Interested candidates should send their CV/resume and a short statement of purpose to Halil Kilicoglu (halil@illinois.edu) and Jodi Schneider (jodi@illinois.edu). Review of applications will begin immediately. Applications will be accepted until the position is filled.

Posted on Handshake; note that Halil is also hiring an NLP Research Assistant (50% RA) and an NLP hourly (10 hours per week) for related work on the same project.

Start date – January 16, 2022

Description, Responsibilities, & Qualifications:
Mixed methods research assistant to Information Sciences faculty. The incumbent will join the Information Quality Lab under the direction of Dr. Jodi Schneider to work on a newly-funded, three year IMLS grant, Strengthening Public Libraries’ Information Literacy Service Through an Understanding of Knowledge Brokers’ Assessment of Technical and Scientific Information. This project will conduct mixed methods case studies—COVID-19 year 1; climate change (year 2); and AI and labor (year 3)—to understand how knowledge brokers such as journalists, Wikipedia editors, activists/advocates, public librarians assess and use scientific and technical information. Ultimately, the project will develop a conceptual model about sensemaking and use of information. Starting in 2023, the team will co-develop services for knowledge brokers and the public, in collaboration with public library test partners. Results from the project will have implications for public access, information literacy, and understanding of science on policy-relevant topics.

Duties may include:

• Synthesizing a collection of existing literature related to knowledge brokers.
• Collecting a sample of about 250 public-facing documents and multimedia, including news (e.g., online print outlets), Wikipedia pages, membership-based online forums, documentaries, and data visualizations, that report, quote, or analyze scientific products (research papers, preprints, datasets, etc.).
• Using topic modeling, argumentation analysis, and other document analysis techniques to analyze documents and multimedia.
• Preparing for and conducting interviews with knowledge brokers (journalists, Wikipedia editors, activists/advocates, public librarians).
  • Developing an interview protocol to solicit information from journalists, Wikipedia editors, activists/advocates, public librarians, etc. to understand how they assess the quality of scientific and technical information.
  • Identifying COVID-19 knowledge brokers to interview, by using the document/multimedia collection, organizational directories, etc.
• Qualitative analysis of interview transcripts (including correcting automatically generated interview transcripts).

Required Qualifications:

• Excellent communication skills in written and spoken English
• Excellent analytical/critical thinking skills and effective time management skills
• Interest in topics such as misinformation, information diffusion, science/technology policy, etc.
• Interest or experience in one or more methods such as: mixed methods, document analysis, altmetrics, semi-structured interviewing, critical incident technique, or qualitative data analysis

Preferred Qualifications:

• Available for multiple semesters, including summer
• Experience conducting and/or transcribing interviews
• Experience with qualitative analysis software such as ATLAS.TI, NVivo, Taguette, RQDA, etc.
• Experience as a journalist, Wikipedia editor, activist, advocate, public librarian, information conduit, or knowledge broker
• Enrollment in the Master’s in Library and Information Science program or in a PhD program
• Previous completion of one or more CITI Program ethics trainings modules
• Experience in academic and/or scientific writing

Application Procedure: Interested candidates should send a cover letter and resume in a single pdf file named Lastname_IMLS_RA.pdf (e.g., Schneider_IMLS_RA.pdf) to ischool-infoquality@illinois.edu

Review of applications will begin immediately. Applications will be accepted until the position is filled. All applications received by November 15, 2021 will receive full consideration.

Posted on the Assistantship Clearinghouse.

Start date – ASAP

Description, Responsibilities, & Qualifications:
Mixed methods research assistant to Information Sciences faculty. The incumbent will join the Information Quality Lab under the direction of Dr. Jodi Schneider to work on a newly-funded, three year IMLS grant, Strengthening Public Libraries’ Information Literacy Service Through an Understanding of Knowledge Brokers’ Assessment of Technical and Scientific Information. This project will conduct mixed methods case studies (first topic: COVID-19) to understand how knowledge brokers such as journalists, Wikipedia editors, activists/advocates, public librarians assess and use scientific and technical information. Ultimately, the project will develop a conceptual model about sensemaking and use of information. Starting in 2023, the team will co-develop services for knowledge brokers and the public, in collaboration with public library test partners. Results from the project will have implications for public access, information literacy, and understanding of science on policy-relevant topics.

This position may become a tuition waiver generating assistantship for the Spring 2022 semester for eligible Master’s and Doctoral students.

Initial duties will include:

• Developing an interview protocol to solicit information from journalists, Wikipedia editors, activists/advocates, public librarians, etc. to understand how they assess the quality of scientific and technical information
• Synthesizing a collection of existing literature related to knowledge brokers
• Collecting a sample of about 250 public-facing documents and multimedia, including news (e.g., online print outlets), Wikipedia pages, membership-based online forums, documentaries, and data visualizations, that report, quote, or analyze scientific products (research papers, preprints, datasets, etc.)
• Identifying COVID-19 knowledge brokers to interview, by using the document/multimedia collection, organizational directories, etc.

Future work will include:

• Conducting interviews with knowledge brokers (journalists, Wikipedia editors, activists/advocates, public librarians)
• Correcting automatically generated interview transcripts
• Qualitative analysis of interview transcripts
• Using topic modeling, argumentation analysis, and other document analysis techniques to analyze documents and multimedia
• Case studies on climate change (year 2) and AI and labor (year 3)

Required Qualifications:

• Excellent communication skills in written and spoken English
• Excellent analytical/critical thinking skills and effective time management skills
• Interest in topics such as: misinformation, information diffusion, science/technology policy
• Interest or experience in one or more methods such as: mixed methods, document analysis, altmetrics, semi-structured interviewing, critical incident technique, or qualitative data analysis

Preferred Qualifications:

• Available for multiple semesters, including summer
• Experience conducting and/or transcribing interviews
• Experience with qualitative analysis software such as ATLAS.TI, NVivo, Taguette, RQDA, etc.
• Experience as a journalist, Wikipedia editor, activist, advocate, public librarian, information conduit, or knowledge broker
• Enrollment in the Master’s in Library and Information Science program or in a PhD program
• Previous completion of one or more CITI Program ethics trainings modules
• Experience in academic and/or scientific writing

Compensation: minimum $18/hour for Master’s students or $20/hour for PhD students (negotiable commensurate with experience)

Application Procedure: Interested candidates should send a cover letter and resume in a single PDF file named Lastname_IMLS_hourly.pdf (e.g., Schneider_IMLS_hourly.pdf) to ischool-infoquality@illinois.edu.

Review of applications will begin immediately. Applications will be accepted until the position is filled. All applications received by November 15, 2021 will receive full consideration.

Posted on the University of Illinois Financial Aid Virtual Job Board and Handshake.