Request for Comment: Publication Ethics for Peaceful Science

Peaceful Science
1

We are seeking feedback on a draft proposal for establishing policies and procedures on error correction. This is the first project of a committee charged with the task of creating policies and procedures on publication ethics at Peaceful Science. The committee’s current membership is @cwhenderson, @Jordan, and @sfmatheson.

To provide suggestions and critiques, or to ask questions, please respond here in the forum. Note that the conversation will be carefully moderated to stay on topic (so that the committee gets clear feedback), and note that while the proposal is focused on a pretty specific topic, an “unanswered questions” section points to future projects and topics on which we welcome input. If you would prefer to provide comments privately, that’s fine–use the private messaging facility here or feel free to contact Stephen Matheson by email (which you can find at sfmatheson.com).

Proposed Policy and Process for Error Correction

Applies to blog posts and articles at peacefulscience.org. Does NOT apply to posts on the discussion forum, which is at discourse.peacefulscience.org

Policy

Pre-publication processes can mitigate errors, and those processes are part of a separate set of principles and standards. But errors will happen, and these basic principles apply whenever an error is noted. Peaceful Science expects and welcomes error correction.

Error correction will be comprehensive, transparent, and public. Even the correction of a typo or misspelling will be noted. Correction mechanisms will range from editing in place (with full disclosure) to retraction.

Categories of error

There are two overall categories of error that can happen in an article.

Minor errors include typos, misspellings, inaccuracies in citations or attribution, grammatical or format-related errors that hinder clarity or obscure the author’s intended meaning. Correcting a minor mistake leaves the intended meaning unchanged.

Errors of fact, whether they seem major or minor, are almost always a separate category from minor errors. Correcting an error of fact may not change the overall message of an article, or affect its conclusions, but the correction changes the meaning of at least a portion of the article.

What is not an error

In the context of ongoing scholarship (science, specifically), articles regularly become “wrong” when new discoveries happen. This is why science is often correctly described as “self-correcting,” and the scientific literature has no other mechanism for correction of this kind of wrongness. The policies in this proposal do not address this kind of ongoing scientific self-correction, but future proposals will.

Processes for correcting errors

Minor errors are corrected in place. This means that the article is edited, and the original version containing the error is replaced. All changes are disclosed in a note attached to the article, and the changes are specifically identified.

Errors of fact result in either correction via a corrigendum, or retraction.

When correction of errors in an article leaves the article’s main points and conclusions intact, a corrigendum is a reasonable process (more details below). There are at least two related factors that can result in the more dramatic step of retraction, and both involve editorial judgment. The first factor is the extent to which the error undermines the article. If an author’s main points rely significantly on false claims, then correction of the error renders the article untenable and retraction is a likely outcome. The second factor is misconduct. If an article is tainted by scholarly misconduct (gross misrepresentation of facts or other scholarship, selective citation, quotemining and other misrepresentation of the words or work of others, and so on), retraction is a likely and reasonable outcome.

A corrigendum is a separate article (or an addendum to the original article) that identifies the error, its source (either the author or, in rare cases, the editor), and the steps taken to correct the error. The corrigendum points to the now-corrected article, and the article points to the corrigendum. The original article can be retained and linked in the corrigendum. If this is done, the original article must be very clearly marked to alert readers to the fact that the article has been corrected.

A retraction notice is attached to the original article and specifies the reasons why the article is being retracted and the parties who have agreed to its retraction. Retracted articles should remain accessible to readers, in a form that very clearly indicates that the article has been retracted and why.

Unanswered questions

This draft document is fully open for discussion. Some very important questions are not addressed, but must be settled (through discussion) before the policy is complete.

  1. Error correction requires oversight and judgment from a person or committee, which is not addressed in this draft.

  2. Editorial judgment regarding type of error (minor error or error of fact) and scholarly conduct requires standards and criteria, which are not specified in this draft.

  3. Pre-publication review is not discussed in this draft.

  4. Retention of articles after correction, through a system of versioning, is hinted at here but not discussed in detail.

  5. Retraction due to misconduct is likely not workable without explicit agreement by authors to be subject to editorial judgment. This draft says nothing about how to navigate that.

  6. Correction of wrongness that results from ongoing discovery is not addressed in this draft. One suggestion is to create a set of “living documents” that comprise summaries or discussions of scientific topics, organized so that the reader can see the development of ideas in time without the “correction” or removal of articles. Another suggestion is to annotate articles with pertinent updates, similar to the process for annotating articles with corrections.

References:

COPE (the Committee on Publication Ethics) sets standards for publication practice. The first reference is a PDF summary document of all aspects of publication practice and covers all the topics in our draft. The second is specifically about retraction.

COPE Retraction Guidelines

COPE Guidelines on Good Publication Practice

3 Likes
3

I want to thank @sfmatheson for chairing this committee, and @jordan and @cwhenderson for serving on it. This draft policy is an important step in the committee’s work, and I expect the committee to take all feedback very seriously, so we can produce a workable policy that feasibly holds us to high standards.

I particularly want to hear feedback from science bloggers, scientists, and journalists familiar with the dialogue, e.g. @MStrauss, @glipsnort, @NLENTS, @art, @sygarte, @ccreyes, @Kristel, @dga471, and @Michelle. Though this policy is for Peaceful Science, we hope that other organizations would adopt it too. We also hope it will be feasible enough that even science bloggers might adopt it too. So your input matters here a great deal, and we look forward to receiving it.

2 Likes
5

Here is some specific feedback for the committee to consider from me.

  1. The policy needs to include a provision about making the policy publicly known, with a process for reporting errors. Such as:

Websites adopting this policy should prominently note this on their website, including a link to the policy and information on how to report errors to the website’s editors. Any deviations from the recommended policy of this document should be clearly noted, ideally with an explanation of why the deviation is needed.

  1. I suggest adding a provision for modifying the policy. This will keep people from vetoing because of a minor misfit, and also give us a path to improve the policy over time. Consider text such as:

We recognize that this is a complex and rapidly changing space, with several publishing niches. It is possible that some provisions of this policy are not suitable in some contexts. Any deviations from this policy should be clearly noted. We encourage adaptations of this policy to be made, and to be communicated back to us, so that revisions of this document can be considered.

  1. Once the first version of this policy is finalized, I suggest maintaining it on github (see: Open collaborative writing with Manubot and Referencing and citing content - GitHub Docs). With this approach, updates to the policy can be publicly logged, issues with the policy can be reported by anyone, and adoption of changes debated and finalized transparently. By including it in github, we can also obtain DOI’s for each version of the policy.

  2. I suggest specifying a copyright license. I suggest selecting the appropriate Creative Commons license (see: About CC Licenses - Creative Commons) and noting at the end of the policy as required by their guidelines.

  3. This point about what is “not an error” needs some clarification.

This is to open to abuse and needs some guidelines, even if a full blown policy is not developed now. Perhaps include some text such as:

There may be debate about what is an error that requires correction. In general, we should err towards clarifying points that will confusing to readers, especially non-scientist readers. Though we do not include a full policy on this here, good faith application of this principle would include adding dated notes to articles that clarify for readers how understanding has advanced.

  1. Some policy on disputes should be included, such as:

There may be disputes or significant ambiguity about whether or not a specific issue in an article is an error, or if the error requires a retraction or not. Editors are expected to use their judgement in making a determination, but also to be transparent about these disputes. This should include noting any unresolved disputes in a note in the article, or possibly in comments on the article, and noting the name of the editor that made the determination.

  1. There seems to be more information necessary in correction or retraction notices:

The correction/retraction notices should also include 1) the name of the person who reported the error (unless they elect to be anonymous), 2) the date the error was reported.

  1. How does this policy apply retroactively? I suggest adding a provision such as:

This policy applies retroactively to all articles published, with the caveat that articles will be brought into compliance as issues are noted to the editorial staff.

  1. Language idiosyncratic to us, e.g. reference to peacefulscience.org, should be removed.
1 Like
6
  1. I suggest a clause on silent deletions/corrections of scientific content:

Articles and pages with scientific content should never be silently deleted or corrected. This is especially important of articles that include scientific content that has been disputed or referenced by others. Articles with dated content (e.g. notice of a job opening) that does not include any scientific content may be deleted or corrected without any notice.

1 Like
7

I agree that we should always be eager to add notes of clarification, updates, pointers to more up-to-date articles, etc. A “full policy” on this need not be complex, unless it is our intention to define ‘error’ as including ‘out of date.’ (That would be a unique and IMO unworkable definition of ‘error.’) And implementation can take the form of dated notes or, more ambitiously, versioning as suggested (for discussion) in the OP.

In my experience in mainstream publishing, the first is never done and I think there are very good reasons for that. The person(s) bringing errors to light should IMO assume that their identities are not relevant and would not be disclosed. If they want to be named/credited, that seems fine to me but it is IMO a potentially problematic precedent to set. The second suggestion is important and I agree.

Since the practice of error correction is always retroactive, the only concern here is whether authors at PS before now might not agree to the policy. A statement about applying policy to past articles seems unnecessary but isn’t problematic as near as I can tell.

All of the other suggestions sound great; most are about the maintenance and publication of the policies themselves and so a bit outside the scope, but important and useful. The comments about disputes are something for the committee to think about; IMO, the potential for abuse there is much greater than in any discussion or policy about out-of-date materials.

1 Like
8

@swamidass, a question/clarification that I wondered about, in terms of setting context for this policy, is are we going to distinguish between types of articles?

For instance, in an article that presents thoughts and opinions rather than something like a scientific analysis or fact-heavy arguments, I feel like fielding “disputes” doesn’t make much sense. I don’t know if we want to get into the habit of fielding requests for “error-correction” on a post like @sfmatheson’s:

So do we need to make any distinction between types of posts (something like the difference between an opinion vs news section in a newspaper)?

3 Likes
9

I agree, and I hope (perhaps unrealistically) that the policy will not be about disputes but about errors. Of course it always requires editorial judgment to decide whether a particular glitch is an error, vs. a typo, vs. a sentence about which people disagree. And we should do our best to identify criteria that distinguish typos from errors of fact from disputed opinions/interpretations. But ultimately, if our openness to correction leads to abuse of the system to adjudicate disagreement, then we’ll need to do more work in our community.

I think we can mention this in the policy, but my view is that we should be open to correcting error no matter where it is. If my piece about humanism contains errors of fact, I will want them to be corrected and our policy/procedures can facilitate that and make it transparent. IMO it shouldn’t matter that I was writing a personal perspective piece–I am still subject to ethical standards and I am obligated to present facts accurately. To me it is axiomatic that an expression of an opinion or the advancing of a view point cannot be challenged on a factual basis, but an opinion piece can still contain errors of fact.

1 Like
10

My primary concern is scientific content, but I do not object to establishing a policy that covers typos and other minor errors. I agree with @sfmatheson that this really covers all content published on the blog, including articles of all types and pages.

It is beyond your scope to develop a full dispute resolution recommendation. However, we are in a contested space, so entirely expect that:

  1. There will be legitimate disputes about what qualifies as a major error.
  2. There will be abuse of any ability to raise disputes.
  3. There will be abuse of editorial judgment, such as avoiding correction by claiming the scientific error is disputed and by ignoring legitimate disputes of the scientific facts.

A good policy will recognize these realities and provide some guidelines to manage this. In general, I think that editors can be granted leeway (and we don’t need to propose a formal dispute resolution process), but as a check/balance disclosure should be required to prevent editors from abusing their power. It is in navigating these conflicting that I am hopeful to receive helpful recommendations from you.

One possibility is to require disclosure of disputes on scientific accuracy if they are brought forward by well-qualified scholars. That requirement of “well-qualified scholars” should be enough to prevent the vast majority of abuse. If all that’s required is disclosure of the dispute, and perhaps clarifications as to the comments to which the dispute pertains, it does not seem like this could be easily abused.

11

This may be a place where it should be different for us in regards to major errors. Let me explain why.

We are in a very contested space. There is a strong tendency to dehumanize and ignore those with whom we disagree. Part of how I believe Peaceful Science is supposed to speak into this is by humanizing our intellectual opponents, and giving them credit where credit is due, especially when we have been found in error.

Take a look at this article, and it’s first paragraph:

I was wrong. I incorrectly used the terms “monophyletic” and “polyphyletic” in my book, The Genealogical Adam and Eve . My intended meaning was clear from context, but departed far from the usual meaning of the words. As biologist Richard Buggs notes in his review of my book:

Confusingly, Swamidass describes his created Adam and Eve as “monophyletic” with evolved human beings, but by this he means “of the same biological type” (p. 85), rather than the more usual definition of “sharing common ancestry”.

Dr. Buggs is entirely correct. I was wrong. Devin Gouvêa and Kenneth Kemp, two philosophers, were also quick to identify this error and I am grateful that they brought this to my attention. This is an error that needs to be corrected.

I could have just made the correction without acknowledging Buggs and the others, especially because Buggs is perceived to be connected to an organization with which many of disagree. I wanted to do better than this. Even though I’ve disagreed with Buggs in the past, he was right here, and I want him to get credit for correcting me by name. That’s because this is such a contested area, and because we disagree on other points.

Now, if this is out of line with standard practice, I still think it should be our practice. It might also merit further explanation, so people understand the values driving this particular policy.

That’s my opinion, but I’m not running the committee though. I do think it is important to add this in, but I am very attentive to your advice on this.

12

@sfmatheson and @swamidass, I think we’re on the same page. I was just thinking ahead a little bit about potential abuse of the policy if we move from “error correction” to “dispute resolution”. I think absolutely that factual errors need to be corrected, regardless of the type of article, and having a transparent policy to do so will serve us well.

2 Likes
13

This is a point where I should give an example. Take a look at this article where I made a good faith error and promptly retracted it:

What if this retraction did not satisfy the guidelines you recommend? Do I go back and do it correctly? Or do I leave it as is? Or do I leave it as is until someone asks us to bring it into compliance?

It seems impossibly difficult to pro-actively implement a policy retroactively on all posts. It would require a full review of our website. We are small, so it is barely feasible, but it would be totally infeasible for others who have larger websites. On the other hand, we want to be as transparent as possible. So I do wonder if the last option is the best: leave past articles “as is” until someone formally requests (on a case by case basis) to bring specific articles into compliance.

In those cases, I do not think it is always necessary to add an additional note stating we edited the article to be in compliance.

14

Perhaps that’s what is needed, to state that we will be eager to add clarifications, updates, and pointers to more up-to-date articles, etc. I do not think we should define error as “out of date,” but I do think there should be some language defining this as a category and noting appropriate remedies to take when we are asked to do so. I don’t think it is reasonable to require proactive effort to take these steps, as that would not be feasible, but we could require good faith effort to take these steps if formally requested.

I do think that thinking through this is important. It does seem that this should be developed somewhat, if we are to take seriously the goal to be a trustworthy source of information to casual readers from the public.

15

The discussion here has been good, and I’m nearly done giving all my feedback on this version. I’m looking forward to hearing the feedback from others in the next week. I did have another point to consider:

  1. The policy should include some guidelines about promptness in making corrections. How quickly should errors be corrected? How should deviations from this time limit be disclosed and explained?
16

That, to me, is an argument for being as clear as possible that a person who notes an error can assume that PS will not use their name or refer to them unless they explicitly ask. The example you provide gave credit to someone who had publicly written about the topic, and so your comments are IMO just a form of important (indeed obligatory) citation to give proper credit. As a principle, this is not different from the ethical obligations of all scholars to properly cite others.

This is not solved by creating an atmosphere in which a person who finds an error must enter into a public scholarly arena. Again, I would aim for something very different and would suggest we be open to anonymous communication about errors. I don’t see how we could do that while creating an expectation of naming people who identify errors. Separately we can be sternly explicit about expectations for scholarly credit, and in fact I have listed those issues under policies (to be developed) about misconduct.

1 Like
17

Well, first, it’s not a retraction and I think it is important to use the terms clearly. You made a correction, and indeed a minor one. (By ‘minor’ I don’t mean ‘unimportant,’ I mean that the correction you made was to a short letter and didn’t IMO undo anything else in the letter.) The correction was important but it was not a retraction. I hope that our proposal will help everyone understand and agree to the distinction.

Second, this is a nice example of how in principle we want to see corrections (corrigenda) work. The error is identified clearly, it is corrected clearly, and both the original document and the correction are linked to each other. In this case, you didn’t do “edit in place” but that seems appropriate to me, since the error is in an open letter, to an individual. If we were to address this case right now, using some version of the policies we are considering, I think we would do exactly what you did.

But I will sign off tonight by asking a favor. We already have a significant list of requests/suggestions, and many of them go outside the scope of what the proposal intended to do, which was this: to begin with some key principles and to establish a framework that we can start with. It was a careful decision to keep the scope limited, because a vast list of ideas is probably not amenable to community discussion and because the committee consists of full-time professionals working as volunteers on a side project during a pandemic. If only as a favor to me, please consider keeping the discussion focused on the proposal itself and its unanswered questions.

1 Like
18

Thanks for the clarifications. I agree with just about all of it. We should receive anonymous reports of errors, and we should be obliged to give scholarly credit.

At this time, I think I’ve raised all the issue I think are important to raise. I trust you guys to parse out what needs to be included in this policy, another policy, or the report itself.

So, yes, I can keep it focused from here. :slight_smile:

1 Like
19

Promptness may depend on how long it takes to resolve the question. A notice about “disputed points” could be made as a placeholder for questions that take longer to resolve.

2 Likes
20

The comments and discussion so far have been great, both helpful and ambitious. The project is a big one and I still think it should be tackled in stages. But this seems an excellent start.

Please continue to provide feedback and suggestions. The “unanswered questions” are there to point us toward some topics/projects that seem important to address soon.

I will be away from Peaceful Science for the next week or two but I will eventually see any discussion here, as will the other committee members. My contact information is easy to find at sfmatheson.com if there’s something you want me to see right away.

2 Likes
21

I’m getting an “IP address not found” error on your website, @sfmatheson.

1 Like