Assessing the quality of information

Structure of the publication

A scientific publication is systematically structured and includes (most of) the following components:

• Title: Contains the official and complete title, along with any subtitle of the publication. Example (pdf, 33 KB)
• Author(s): Includes the official name or names of the author(s) of the publication. Example (pdf, 28 KB)
• Abstract and/or Keywords: A brief summary or abstract of the publication, sometimes accompanied by keywords or terms related to the publication. Note: Not all scientific publications have abstracts. Example (pdf, 30 KB)
• Introduction: Provides a brief description of the problem statement, sometimes (but not always, and not necessarily) the methodology used, and the structure of the publication. Example (pdf, 55 KB)
• Main Content (Corpus): Contains the actual elaboration of the problem statement, possibly with accompanying tables and/or graphs. Example (pdf, 581 KB)
• Conclusion: Presents a concise overview summarizing the main research findings. Example (pdf, 43 KB)
• Footnotes or Endnotes: Include brief explanations that do not fit in the main text (for example, because they would disrupt the logical flow of the argumentation). In some referencing styles, footnotes or endnotes also include references to other scientific literature (e.g., in abbreviated form, with full bibliographic information found in the bibliography). Example (pdf, 337 KB)
• Bibliography (Literature List): Provides an overview of the used information sources, formatted according to a consistently applied referencing style. In referencing styles where all bibliographic information is already provided in detail through footnotes or endnotes, a systematic literature list may not be included at the end of the document. Example (pdf, 96 KB)

Content of the publication

Every author writes an article or book from a certain perspective and with a specific purpose. This perspective can be purely scientific, but there may also be other motives and interests at play.

Every piece of information (publication) is always colored by the information provider (author). This, in itself, is not a problem, but sometimes this coloring can lead to a (gross) distortion of the facts.

A (gross) distortion means that information is deliberately twisted or concealed with the aim of leading the reader to the conclusion the author presents. You can recognize this manipulation by a number of characteristics:

• Argumentation: The author bombards the 'opposing party' with personal or irrelevant arguments or ignores the arguments of the 'opposing party.'
• Data: Verifiable evidence is lacking (e.g., numerical data without source references).
• Language Use: The author frequently employs emotional style and language.

To form your own judgment about the quality of the information found, the technique of critical reading is important. In principle, there are two types of text-critical arguments: internal and external:

• Internal arguments: Internal arguments involve identifying inconsistencies and imperfections within the text itself. These may include illogical reasoning, unjustified correlations, unfounded equivalences or contradictions. It can also involve assertive statements without evidence or concrete examples, or quotations taken out of context.
• External arguments: External arguments require the reader to possess knowledge not found within the text itself. This is the crucial aspect of critical reading: you must assess whether a text presents an accurate representation of the facts, and for this, you need broader knowledge than the text provides.

Age of the publication

An older publication is not necessarily outdated. However, it may be less reliable when:

• The research domain is intensely studied, leading to rapid emergence of additional, corrective, or new information.
• The subject itself is temporary or time-bound (e.g., economic recession, language usage among teenagers).

Therefore, it is always a good idea to seek recent publications on your topic, along with the most important older publications.

Note: In most disciplines, it often takes months (for articles) or even years (for books) before researchers see their publications appear. As a result, the published information is often older than the publication date suggests.

Profile of the Author

An important criterion for assessing the reliability of information is the profile of the author. There are four types of qualifications on which you can evaluate the profile of an author:

• Education: To obtain a university degree, an author must have demonstrated their expertise in the field of study.
  There are different levels of education: a doctorate is a higher qualification than a bachelor's or master's degree.
• Position: Authors with important leadership positions may not have the highest educational qualifications, but some possess extensive experience that makes them credible and reliable.
• Affiliation with a reliable institution: Researchers gain credibility through their affiliation with a university or other reputable scientific institution. However, it is important to note that employees of such institutions are not necessarily the most competent researchers in their field.
• Hirsch Index: The H-index is an indicator of the research output and scientific impact of authors. If a researcher has an H-index of 15, it means they have published 15 journal articles that have been cited at least 15 times by other researchers. A higher H-index indicates a greater impact.
  Please note: the citation reports from Web of Science (one of the databases where the H-index is automatically calculated) are not entirely error-free. Moreover, the value of the calculation in Web of Science is less relevant for researchers in the humanities because journals in these disciplines are less frequently included in this collection.

Important! These qualifications do not provide authors with credibility in all domains but are only applicable to their area of expertise. For instance, a professor of musicology may not necessarily possess the appropriate qualifications to make reliable statements about the Holocaust.

Assessment beforehand: Editorial boards and peer review

Before a publication is accepted by a scientific publisher, it must undergo evaluation. This evaluation can be carried out by members of the editorial board or by specialized fellow scientists.

• Editorial boards: The members of these editorial boards must be experts in the specific scientific domain to make a reliable judgment about the submitted publication.
• Peer review: Some editorial boards systematically seek the assistance of fellow scientists who are specialized in the relevant field. This external evaluation is known as peer review. As a rule of thumb, the more prestigious the journal, the stricter the peer review process.

To prevent factors other than the content from influencing the evaluation, the names of the authors are often hidden from the colleagues who conduct the evaluation, and vice versa (double-blind peer review). Because this anonymity can also result in less constructive feedback, the system is under pressure. In some journals, reviewers have the option to reveal their identity after publication, thus acknowledging their often unpaid efforts.

Assessment afterwards: Reviews and Citations

Every publication is part of a scientific network and is evaluated by experts. You can find these evaluations through reviews or citations.

• Reviews (book): A review is a critical evaluation of a scientific study by an independent scholar familiar with the subject matter. Reviews of recently published books are often compiled and presented in a regular section of a scientific journal. Other terms for a review include 'boekbespreking' (N) and 'compte rendu' (F).
• Citations: Citations or references in footnotes can also help measure the reliability and quality of a publication.
  • Qualitative: If the scientific results are incorporated into new publications, it enhances the authority and reliability of the original publication.
  • Quantitative: A large number of references suggests that this publication has had a significant impact. However, it does not in itself provide sufficient evidence of the reliability of the publication. It could also mean, for instance, that the scientific network is strongly reacting to a controversial proposition in this publication.

Article Assessment: Check the Citation Score

How many times have other researchers cited this article? The more it has been cited, the more value is attached to this article. However, keep in mind that relatively new articles may not have had the chance to establish themselves.

Below is an example of a citation score in Web of Science. You can see the number of times this article has been cited in the Web of Science Core Collection and the total number of citations in all databases of Web of Science.

Please note: These numbers are not absolute; only the citations indexed in Web of Science are shown. Therefore, it is possible that this article is also cited elsewhere. For this reason, access via Web of Science, especially for articles in the humanities, may have limitations.

Assessment of a Journal: Check the Impact Factor

Does the journal have an impact factor, and how high is it? The impact factor is a measure of the frequency with which an "average article" in a journal is cited in a specific year or period. A high impact factor indicates that an average article in the journal is cited relatively frequently. By comparing journals within a specific field, it becomes clear which journal has a higher impact factor.

You can find impact factors in the Journal Citation Reports, and increasingly, they are also available in the editorial section of a journal.

Example of a journal with an impact factor in the Journal Citation Reports:

• Go to the website of the Journal Citation Reports. Enter the title of the journal in the search bar and click on the magnifying glass.

• In the Search Results, click on the title.

• A table with the impact factor and data about the journal's impact over the past five years will appear.

Below are some quality criteria summarized in key terms for the substantive evaluation of website quality.

Reliability

Is the content of the source well-founded and accurate?

• Is the information objective and based on sound research?
• Are there references or a substantial bibliography?
• Does another format of the source exist that is known for its quality and reliability?
• If there is any bias, is it openly acknowledged and of an acceptable standard?
• Has visible care been taken in compiling the information?

Pay special attention to:

• Literature citations and bibliographies
• Justification of the source of information
• Mentioning the purpose of the website
• Typographical, spelling, and grammatical errors
• Extensions: Websites from educational institutions (US: .edu / UK: .ac.uk), non-profit organizations (.org), and governments (US: .gov) are generally more reliable than commercial sites (.com).

Currency

Is the content of the source up-to-date?

• Is the substantive information well-maintained?
• Is newly available information added promptly?

Pay special attention to:

• The date the information was created or made available ('creation date' or 'publication date').
• The date the information was last updated ('last updated date').
• Any notification regarding update or publication frequency.
• The availability of archived information.

Status

Authority:

• Who publishes this information on the internet?
• Is the source known to be authentic and authoritative?
• What is the status of the author and/or the publisher?

Pay special attention to:

• Additional information about the author and publisher (possibly under links such as 'about us,' 'contact our company,' etc.).
• Further justification of the sources of information from which certain presented facts and data are derived.

Tip: To assess the importance or authority of the website, you can use Google's link filter to check how many external websites refer to the respective source website. For example, if you type link:UHasselt.be, you will get a list of all websites that refer to the UHasselt website. However, keep in mind that, more than with other publications, the number of references to a website is not an absolute guarantee of reliability.

Completeness

What is the scope and depth of the information?

• Is there sufficient depth in the coverage of the subject?
• Are the various aspects of the subject adequately addressed?
• Are there identifiable gaps in the information?

Pay special attention to:

• A table of contents.
• An index.
• A site map.
• Literature citations and bibliographies.

Suppose that you use Google to search for the birth date of Sir Isaac Newton. When you carefully examine the results, you notice that different websites mention different dates. Some sites state 1642 as the birth year, while others mention 1643.

Which of the two found birth years is correct? In this case, you quickly realize that the difference is due to calendar changes. When Newton was born, it was indeed December 25, 1642. However, according to the current calendar system, he would have been born on January 4, 1643.

When you encounter conflicting information, it is best to try to find a reliable website that mentions both options and explains which one is correct or why they differ.

Some guidelines:

• Consult other websites and (printed) sources such as encyclopedias, textbooks, etc.
• Keep in mind the purpose for which you are seeking information. If you are looking for the main theories of Isaac Newton, his exact birth year might not be crucial, but it becomes more important for his biographical account.
• Contradictory information is not always avoidable. Even scientists may contradict each other at times.
• Remain critical. You will need to use your own analysis and insight to determine which information to accept and which not to.

Wikipedia is the world's largest online internet encyclopedia, freely accessible and editable by anyone. It is important to exercise caution regarding the accuracy of the information provided.

As with all sources where the author is unknown, you should consult a second source to assess the quality of the information.

Many Wikipedia articles include references at the end of the text, indicating the sources from which the information was derived. You can use these original sources for your paper. If no sources are mentioned, it is not advisable to rely on the text from Wikipedia.