Posts Tagged ‘annotation’

Medical/Life Sciences Graduate or Undergraduate Student Hourly – Biomedical Literature Annotation for Citation Accuracy/Integrity (10 hours per week, spring semester) – School of Information Sciences – University of Illinois at Urbana-Champaign

January 13th, 2023

The ScienceNLP Lab and the Information Quality Lab at the School of Information Sciences (iSchool) are seeking a University of Illinois at Urbana-Champaign student to read and annotate health-related literature. Knowledge and training and knowledge in a medical/life sciences field such as biology, chemistry, bioinformatics, food science and nutrition, or bioengineering is essential. The hourly will work an average of 10 hours per week for spring semester, under the co-supervision of Dr. Halil Kilicoglu and Dr. Jodi Schneider. The project focuses on assessing biomedical publications for citation accuracy and integrity. Your role in this project will be to locate citation statements in biomedical articles and assess their accuracy with respect to the cited articles. You will collaborate with other annotators on this task. This work is part of the project Natural Language Processing to Assess and Improve Citation Integrity in Biomedical Publications, funded by the Office of Research Integrity (ORI).

Project Description: While citations play a fundamental role in scientific knowledge diffusion and research assessment, they are often inaccurate (e.g., citation of non-existent findings), undermining the integrity of scientific literature and distorting the perception of available evidence. A recent meta-analysis showed that 25.4% of medical articles contained a citation error. A bibliometric analysis revealed that inaccurate citations of a letter published in 1980 may have contributed to the opioid crisis. The project will develop and validate resources and models that aid stakeholders in assessing biomedical publications for citation accuracy and integrity. The publicly available annotated corpus you help create will be used to develop natural language processing/artificial intelligence (NLP/AI) models for assessing reporting quality in biomedical articles.

Duties include:

  • Reading and annotating biomedical publications for citation integrity/accuracy
  • Contribution to development of annotation guidelines
  • Contribution to scientific presentations and publications

Required qualifications:

  • Background in a field such as: medicine, life sciences, including biology, chemistry, bioinformatics, food science and nutrition, bioengineering, or a related field.
  • Excellent English reading comprehension skills
  • Excellent communications skills in written and spoken English
  • Excellent analytical/critical thinking skills
  • Effective time management skills, attention to detail

Preferred qualifications:

  • Interest in topics such as trustworthy science, research rigor/quality, reproducibility
  • Interest in biomedical data science, bioinformatics, or related fields
  • Availability for multiple semesters

Interested candidates should send their CV/resume and a short statement of purpose drawing attention to their training in medicine or life sciences (e.g., biology, chemistry, bioinformatics, food science and nutrition, bioengineering, or a related field) to Halil Kilicoglu (halil@illinois.edu) and Jodi Schneider (jodi@illinois.edu). Review of applications will begin immediately. Applications will be accepted until the position is filled.

Posted on Handshake and the Virtual Job Board.

Tags: , , , , , , , , , , , , , ,
Posted in Information Quality Lab news | Comments (0)

Graduate Hourly – Annotation (10 hours per week, up to 9 months) – School of Information Sciences – University of Illinois at Urbana-Champaign

June 29th, 2022

The ScienceNLP Lab and the Information Quality Lab at the School of Information Sciences (iSchool) are seeking a University of Illinois at Urbana-Champaign graduate student to read and annotate health-related literature. The graduate hourly will work an average of 10 hours per week for up to 9 months, under the co-supervision of Dr. Halil Kilicoglu and Dr. Jodi Schneider. The project focuses on assessing biomedical publications for citation accuracy and integrity. Your role in this project will be to locate citation statements in biomedical articles and assess their accuracy with respect to the cited articles. You will collaborate with other annotators on this task. This work is part of the project Natural Language Processing to Assess and Improve Citation Integrity in Biomedical Publications, funded by the Office of Research Integrity (ORI).

Project Description: While citations play a fundamental role in scientific knowledge diffusion and research assessment, they are often inaccurate (e.g., citation of non-existent findings), undermining the integrity of scientific literature and distorting the perception of available evidence. A recent meta-analysis showed that 25.4% of medical articles contained a citation error. A bibliometric analysis revealed that inaccurate citations of a letter published in 1980 may have contributed to the opioid crisis. The project will develop and validate resources and models that aid stakeholders in assessing biomedical publications for citation accuracy and integrity. The publicly available annotated corpus you help create will be used to develop natural language processing/artificial intelligence (NLP/AI) models for assessing reporting quality in biomedical articles.

Duties include:

  • Reading and annotating biomedical publications for citation integrity/accuracy
  • Contribution to development of annotation guidelines
  • Contribution to scientific presentations and publications

Required qualifications:

  • Excellent English reading comprehension skills
  • Excellent communications skills in written and spoken English
  • Excellent analytical/critical thinking skills
  • Effective time management skills, attention to detail

Preferred qualifications:

  • Background in a field such as:
    • life sciences/medicine, including biology, chemistry, bioinformatics, foodscience/nutrition, bioengineering, or a related field
    • library/informationsciences
    • linguistics
  • Interest in topics such as trustworthy science, research rigor/quality, reproducibility
  • Interest in biomedical data science, bioinformatics, or related fields
  • Availability for multiple semesters

Interested candidates should send their CV/resume and a short statement of purpose to Halil Kilicoglu (halil@illinois.edu) and Jodi Schneider (jodi@illinois.edu). Review of applications will begin immediately. Applications will be accepted until the position is filled.

Posted on Handshake; note that Halil is also hiring an NLP Research Assistant (50% RA) and an NLP hourly (10 hours per week) for related work on the same project.

Tags: , , , , , , , ,
Posted in Information Quality Lab news | Comments (0)

Annotation summaries: standardization needed

August 4th, 2011

I’m finding an iPad amazing for reading PDFs — it’s like instant printing, with no weight to carry around (heavy, and they get wet). And with software like iAnnotatePDF and GoodReader, I can annotate with just a bit more effort than while using pen and paper.

iAnnotate (video review) is the killer app that convinced me to buy an iPad. But it has a killer flaw: I couldn’t keep my reading organized with it.

Hence I started looking into reference managers that would work well on the iPad–allowing annotation, making it easy to keep PDF’s organized, and ensuring that annotations were kept in a sensible place.

Sente fulfills many of my requirements. Sync seems to work effortlessly — well exceeding my experience with other products. The annotation process is reasonably smooth but so far I haven’t found a way to export annotations directly.

This is a bit problematic because PDF editors don’t seem to play nice with each others’ annotations. For instance, iAnnotate and GoodReader both export annotations for their own software. You get something very useful and readable like this:

Page 1, Highlight (Yellow):
Content: “The scientific use of Twitter has received some attention in previous work: [4] and [5] have performed several automatic analyses of tweets collected for different conference hashtags, including for example time series and lists of most active twitterers. [3] and [9] have furthermore carried out manual analyses of tweet contents for conference tweet datasets to determine, what conference participants are tweeting about. [10] are develop ing automatic methods for extracting semantic information from conference tweets. [6] have focused on tweets published by a set of manually identified scientists and have investigated their citation behavior.”

Page 1, Highlight (Yellow):
Content: “citations and references are two sides of the same coin.”

But when you annotate in one program and get notes from another program, things get messier.

For PDFs annotated externally, iAnnotate lists highlights without only grabs text from the notes, like this:

Page 1, Highlight (Custom Color: #fdf7bc):

Page 2, Highlight (Custom Color: #fdf7bc):

Page 2, Note (Custom Color: #fdffaa):
Not sure why this stands out from other lists by individuals.

GoodReader plays a bit nicer with annotations from other programs: it breaks annotations made by other programs at line boundaries. This makes summaries a little difficult to read, but at least there’s some content:

Highlight (color #FDF7BC):
first of all it will have to start with the general problem in

Highlight (color #FDF7BC):
analyzing scientific impact of Twitter:

Highlight (color #FDF7BC):
[6] define

Highlight (color #FDF7BC):
tweet to a peer-U

I’m currently checking into the standardization around annotations summaries.

I’d be very interested to hear about how you detect metadata and annotation differences in PDFs. As examples, I’ve marked up a recent WebSci poster, with some annotations from GoodReader, from iAnnotatePDF, and from Sente.

Tags: , , ,
Posted in books and reading, iOS: iPad, iPhone, etc. | Comments (1)

Papers2 does not integrate with external iPad applications in the way I expected

July 31st, 2011

Papers2 does not integrate with external iPad applications in the way I expected. I use iPad applications like GoodReader, iAnnotatePDF, and PDFExpert to read and annotate papers.

The functionality I expected was:

  • Export from Papers to an external PDF annotation application
  • When I reopen Papers, the annotated PDF is shown in my library

However, here is what happens:

  • Export from Papers to an external PDF annotation application. It renames the file, using a random string as the filename.
  • When I reopen Papers, only the original (unannotated) PDF is in my library.
  • Alternately when I export from the external application, the annotated file is imported as a *new* PDF, unconnected to the original, with a random string used for the filename.

I started using Papers because managing filenames in iAnnotate wasn’t working: I couldn’t figure out which files were which. So this is absolutely key for me.

==

This is a bug report to Papers2, copied here since bug reports are private. Any workarounds or suggestions for alternate annotation/reference management workflows would be very welcome.

This annotation environment completely failed to meet my expectations: I expected to ‘Open In’ an annotation application; in fact there’s just ‘Export’ and ‘Import’, meaning that the annotated file isn’t automatically stored in the Papers2 library.

Tags: , , , , , , ,
Posted in books and reading, iOS: iPad, iPhone, etc. | Comments (1)

Extended deadline for STLR 2011

April 29th, 2011

We’ve extended the STLR 2011 deadline due to several requests; submissions are now due May 8th.

JCDL workshops are split over two half-days, and we are lucky enough to have *two* keynote speakers: Bernhard Haslhofer of the University of Vienna and Cathy Marshall of Microsoft Research.

Consider submitting!

CALL FOR PARTICIPATION
The 1st Workshop on Semantic Web Technologies for Libraries and Readers

STLR 2011

June 16 (PM) & 17 (AM) 2011

http://stlr2011.weebly.com/
Co-located with the ACM/IEEE Joint Conference on Digital Libraries (JCDL) 2011 Ottawa, Canada

While Semantic Web technologies are successfully being applied to library catalogs and digital libraries, the semantic enhancement of books and other electronic media is ripe for further exploration. Connections between envisioned and emerging scholarly objects (which are doubtless social and semantic) and the digital libraries in which these items will be housed, encountered, and explored have yet to be made and implemented. Likewise, mobile reading brings new opportunities for personalized, context-aware interactions between reader and material, enriched by information such as location, time of day and access history.

This full-day workshop, motivated by the idea that reading is mobile, interactive, social, and material, will be focused on semantically enhancing electronic media as well as on the mobile and social aspects of the Semantic Web for electronic media, libraries and their users. It aims to bring together practitioners and developers involved in semantically enhancing electronic media (including documents, books, research objects, multimedia materials and digital libraries) as well as academics researching more formal aspects of the interactions between such resources and their users. We also particularly invite entrepreneurs and developers interested in enhancing electronic media using Semantic Web technologies with a user-centered approach.

We invite the submission of papers, demonstrations and posters which describe implementations or original research that are related (but are not limited) to the following areas of interest:

  • Strategies for semantic publishing (technical, social, and economic)
  • Approaches for consuming semantic representations of digital documents and electronic media
  • Open and shared semantic bookmarks and annotations for mobile and device-independent use
  • User-centered approaches for semantically annotating reading lists and/or library catalogues
  • Applications of Semantic Web technologies for building personal or context-aware media libraries
  • Approaches for interacting with context-aware electronic media (e.g. location-aware storytelling, context-sensitive mobile applications, use of geolocation, personalization, etc.)
  • Applications for media recommendations and filtering using Semantic Web technologies
  • Applications integrating natural language processing with approaches for semantic annotation of reading materials
  • Applications leveraging the interoperability of semantic annotations for aggregation and crowd-sourcing
  • Approaches for discipline-specific or task-specific information sharing and collaboration
  • Social semantic approaches for using, publishing, and filtering scholarly objects and personal electronic media

IMPORTANT DATES

*EXTENDED* Paper submission deadline: May 8th 2011
Acceptance notification: June 1st 2011
Camera-ready version: June 8th 2011

KEYNOTE SPEAKERS

PROGRAM COMMITTEE

Each submission will be independently reviewed by 2-3 program committee members.

ORGANIZING COMMITTEE

  • Alison Callahan, Dept of Biology, Carleton University, Ottawa, Canada
  • Dr. Michel Dumontier, Dept of Biology, Carleton University, Ottawa, Canada
  • Jodi Schneider, DERI, NUI Galway, Ireland
  • Dr. Lars Svensson, German National Library

SUBMISSION INSTRUCTIONS

Please use PDF format for all submissions. Semantically annotated versions of submissions, and submissions in novel digital formats, are encouraged and will be accepted in addition to a PDF version.
All submissions must adhere to the following page limits:
Full length papers: maximum 8 pages
Demonstrations: 2 pages
Posters: 1 page
Use the ACM template for formatting: http://www.acm.org/sigs/pubs/proceed/template.html
Submit using EasyChair: https://www.easychair.org/conferences/?conf=stlr2011

Tags: , , , , , , , , , , , , ,
Posted in future of publishing, library and information science, PhD diary, scholarly communication, semantic web, social semantic web | Comments (2)