import/export to gitlab is broken at HEAD due to upstream changing the message format

Labels: area/bridge/gitlab kind/bug priority/important-soon

sudoforge added label kind/bug

sudoforge added label area/bridge/gitlab

sudoforge added label area/test

sudoforge removed label area/bridge/gitlab

sudoforge removed label area/test

sudoforge added label area/bridge/gitlab

sudoforge added label priority/important-soon

sudoforge commented (edited)

i started on this, but will be AFK for a few days so i won't be able to complete it. if you're interested in the status: there's a detailed comment in the commit; take a look at the PR: https://github.com/git-bug/git-bug/pull/1370

sudoforge changed the title from import/export to gitlab is broken at HEAD due to upstream changing the message format to import/export to gitlab is broken at HEAD due to upstream changing the message format

sudoforge commented

i'll summarize the root cause here for people who'd rather not read the thread on gitlab (there's a bit of back and forth about how to potentially improve this scenario, if you're interested).

when importing from gitlab, in order to get the historical events related to a gitlab issue, we hit the "issue notes api", and get a list of objects that look like this:

{
  "id": 123412341234,
  "type": null,
  "body": "changed title from **bug title edited** to **bug title edited{+ again+}**",
  "author": {
    ...
  },
  "created_at": "2024-03-19T20:56:59.456Z",
  "updated_at": "2024-03-19T20:56:59.463Z",
  "system": true,
  "noteable_id": 12341234,
  "noteable_type": "Issue",
  "project_id": 12341234,
  "resolvable": false,
  "confidential": false,
  "internal": false,
  "imported": false,
  "imported_from": "none",
  "noteable_iid": 1,
  "commands_changes": {}
}

the critical field here is the body field. this value represents that the title was changed from bug title edited to bug title edited again.

gitlab uses this sort of message to render the "comments" that are added to issues when different event types occur: a title being changed, a mention from somewhere else, commits being added, etc. this comprises the "notes" that get added to an issue's feed (you can see a similar thing on github issues, above, for example). for this use case, gitlab doesn't care about the body field's value -- they just dump it into SomeRenderer that spits out something for their UI.

conversely, git-bug cares about the type of event. we track these events as strongly typed "operations", and therefore, we need to parse this event.

historically, this field's value looked like what you see above. with the advent of the 17.11 release, however, gitlab changed the format of the value.

this means that the following two values represent the exact same title change, from bug title edited to bug title edited again:

original

changed title from **bug title edited** to **bug title edited{+ again+}**

post-17.11

<p>changed title from <code class=\"idiff\">bug title edited</code> to <code class=\"idiff\">bug title edited<span class=\"idiff left right addition\"> again</span></code></p>

the kicker here is that this only impacts newly created issue notes. older issue notes still use the original markdown-esque format.

as can be seen from this issue, the approach we take to determine the type of "note" (rather, "event" as we think about it) is pretty fragile, and relies on a loosely typed value that isn't part of gitlab's API contract. unfortunately, there doesn't appear to be another way for us to get this information -- although it's possible gitlab could add support for our use case in the future.

in the short term, we'll have to conditionally use the new parser i wrote in #1370, falling back to the older parser. it doesn't seem like there's an exact timestamp we can use for determining if the body field will be in the new or old format, so the best course of action will just be a brute force approach using a regex.

sudoforge closed the bug