Skip to Main Content

ChatGPT and Generative Artificial Intelligence (AI) for Students

Introduces generative AI and discusses responsible use

Incorrect or imaginary information

The issue

When prompted, ChatGPT may generate text that includes accurate-looking citations. However, it is important to verify the existence of the sources and the accuracy of the information obtained from them.

In the following example, ChatGPT produced a list of three references, of which only the second one exists.

Example: incorrect bibliographic references

ChatGPT screen capture 1. Transcript available.

Fig. 1. OpenAI's ChatGPT, response to query from author, February 7, 2023.

 

Identifying errors and false citations

To verify the existence of the bibliographic references provided by ChatGPT, we can use different approaches:

  • Using Omni, the Library’s catalogue, search for each of the listed references. An advanced search will let you combine different search criteria (for example: author, title, publisher, and year of publication).
    • You can check our research guide ‘What is Omni?’ to learn more about using Omni for finding resources.
  • Using a search engine to look for the book or article. Although the Library collects a large number of authoritative resources, and Omni allows you to search items owned by other academic libraries in Ontario, it is possible that some sources may not be captured in our catalogue, for instance archival primary sources or various forms of grey literature.
  • Asking ChatGPT for additional details related to a particular citation. Example:

ChatGPT screen capture 2. Transcript available.

Fig. 2. OpenAI's ChatGPT, response to query from author, February 22, 2023.

ISBN (International Standard Book Number) and ISSN (International Standard Serial Number) are international numbers that uniquely identify a book or journal. Using this information, you can use Omni or other tools, such as ISBN search, to find the resource being referred to. In the example above, although the ISBN provided by ChatGPT exists, it corresponds to a different title, in this case Sustainable Development Goals: Law, Theory, and Implementation. The publication year is also different than the one in the citation.

 

Additional considerations

  • In the above example, one of the references provided by ChatGPT actually exists: (Intergovernmental Panel on Climate Change, 2014) and we were able to find it using a search engine. However, the existence of this source does not mean that the information presented in ChatGPT’s response comes from this report, or that it has been properly understood and summarized.
  • In this case, ChatGPT may have helped you find one possible source for your work, but you would still need to consult the material to determine its suitability for the research question you are trying to answer, and critically examine its contents in the context of other sources that you have consulted.

Screen capture transcripts: incorrect bibliographic references

ChatGPT screen capture 1 transcript

[Prompt]

Summary of gender based violence and climate justice and climate change with citations in APA format

[Reply]

Here is a summary of the interconnection between gender-based violence, climate change, and climate justice, with citations in APA format:
Gender-based violence (GBV) and climate change are interconnected issues that have a disproportionate impact on women and girls (Hossain & Price, 2019). Climate change, including natural disasters and resource scarcity, exacerbates existing gender inequalities and increases the risk of GBV (Intergovernmental Panel on Climate Change, 2014). Climate justice, on the other hand, refers to the equitable distribution of the benefits and burdens of actions taken to address climate change (Stern, 2018). In the context of GBV, climate justice requires consideration of the specific experiences and needs of women in decision-making and response to the impacts of climate change (Hossain & Price, 2019).
References:
Hossain, M., & Price, L. (2019). Climate justice and gender-based violence. In Handbook of Climate Justice (pp. 497-512). Edward Elgar Publishing.
Intergovernmental Panel on Climate Change (IPCC). (2014). Climate Change 2014: Impacts, Vulnerability and Exposure. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change.
Stern, N. (2018). Climate change and development. Routledge.

ChatGPT screen capture 2 transcript

[Prompt]

When I asked for a summary of the interconnection between gender-based violence, climate change, and climate justice, with citations in APA format, you provided the following reference: 'Hossain, M., & Price, L. (2019). Climate justice and gender-based violence. In Handbook of Climate Justice (pp. 497-512). Edward Elgar Publishing.' Could you please provide an ISBN or ISSN for this publication?

[Reply]

The ISBN for the book "Handbook of Climate Justice," which includes the chapter "Climate Justice and Gender-Based Violence" by Hossain and Price, is 9781786438751. The book was published by Edward Elgar Publishing in 2019.

 

False and outdated information

The issue

ChatGPT may respond to prompts with information that is false and/or outdated. Just as you would with any research, it is important to fact-check and use additional, authoritative sources of information before taking ChatGPT’s answer as definitive. Special care and attention should be paid when doing research whose results may have harmful impact on people, for example, Pharmacy and Health Sciences.
As of this writing, ChatGPT’s training corpus is largely built on texts produced before 2021. As a result, the information obtained may be incomplete and/or factually incorrect, especially in fields of study where new research is produced at a quick pace.

Example: false and outdated information

NOTE: The following example is meant to illustrate some of the problems in using ChatGPT for research in Pharmacy. It should not be taken as medical or pharmaceutical advice

ChatGPT screen capture 1. Transcript available.

Considerations

In this example there is incorrect information:

Screen capture transcripts: false and outdated information

ChatGPT screen capture 1 transcript

[Prompt]

Compare cibinqo to dupixent for the treatment of atopic dermatitis and provide full bibliographic references in the AMA Style.

[Reply]

Cibinqo and Dupixent are both FDA-approved medications for the treatment of atopic dermatitis, a chronic inflammatory skin condition. However, they have different mechanisms of action and clinical outcomes.
Cibinqo (bimagrumab) is a monoclonal antibody that targets and inhibits the activin type II receptor, which is involved in the regulation of muscle mass and inflammation. In a randomized, double-blind, placebo-controlled Phase 2b trial, Cibinqo demonstrated a significant reduction in the Eczema Area and Severity Index (EASI) score, a measure of disease severity, compared to placebo in patients with moderate to severe atopic dermatitis. However, the trial also reported a high incidence of injection site reactions and adverse events, including upper respiratory tract infections, headache, and diarrhea.
On the other hand, Dupixent (dupilumab) is a monoclonal antibody that blocks the interleukin-4 receptor alpha (IL-4Rα) and interleukin-13 (IL-13) pathways, which play a key role in the pathogenesis of atopic dermatitis. Dupixent has demonstrated significant efficacy in reducing the severity of atopic dermatitis symptoms in several randomized, double-blind, placebo-controlled Phase 3 trials. In these studies, Dupixent was shown to significantly reduce the EASI score, improve the overall disease severity, and improve quality of life compared to placebo, with a favorable safety profile.
Overall, while both medications show promise in the treatment of atopic dermatitis, Dupixent appears to be the more effective and safer option based on current clinical evidence.
References:

  1. Ruzicka T, Hanifin JM, Furue M, et al. Anti-interleukin-31 receptor A antibody for atopic dermatitis. N Engl J Med. 2017;376(9):826-835. doi:10.1056/NEJMoa1606490
  2. Simpson EL, Akinlade B, Ardeleanu M, et al. Two Phase 3 Trials of Dupilumab versus Placebo in Atopic Dermatitis. N Engl J Med. 2016;375(24):2335-2348. doi:10.1056/NEJMoa1610020
  3. Nemoto O, Furue M, Nakagawa H, et al. Bimagrumab, an activin receptor type II inhibitor, improves moderate-to-severe atopic dermatitis: a randomized, double-blind, placebo-controlled, phase IIb trial. J Am Acad Dermatol. 2020;83(2):392-400. doi:10.1016/j.jaad.2020.03.094
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Except where otherwise noted, the content in these guides by Mohawk Valley Community College Libraries is licensed under CC BY NC SA 4.0. This openly licensed content allows others to cite, share, or modify this content non-commercially, with credit to MVCC Libraries.

When reusing or adapting this content, include this statement in the new document: This content was originally created by Mohawk Valley Community College Libraries and shared with a CC BY NC SA 4.0 license.

Mohawk Valley Community College Libraries - 1101 Sherman Drive, Utica, NY, 13501- P. 315.792.5561