Multiple bugs with gitlab bridge.

Labels: area/bridge kind/bug

Timeline

Adrien D. (dtrckd) opened (edited)

Hi,

Playing with git-bug on a existing gitlab repository with a few existing issues on the vendor's bug tracker, I ran trough the following errors:

After having successfully configured the bridge I could import the gitlab issues as well as creating a couple of local issues with git bug add, so far so good. Then, I meet the following errors:

  • when doing again git bug bridge pull, issues are duplicated with new hashes.
  • nothing is exported when doing git bug bridge push, it jus returns exported 0 issues with myBridge bridge.

I've tested the bridge on a freshly created repo on gitlab, again the first steps goes well, but this time when doing git bug bridge pull a second time to test if duplicates arise, I got this crash:

panic: runtime error: index out of range [-1]                                                                                                        
                                                                        
goroutine 51 [running]:       
github.com/MichaelMure/git-bug/bridge/gitlab/iterator.(*issueIterator).Value(...)
        /home/dtrckd/src/_sink/git-bug/bridge/gitlab/iterator/issue.go:38
github.com/MichaelMure/git-bug/bridge/gitlab/iterator.(*Iterator).NextIssue(0xc0010ba000, 0xc000043dd0)
        /home/dtrckd/src/_sink/git-bug/bridge/gitlab/iterator/iterator.go:86 +0x244
github.com/MichaelMure/git-bug/bridge/gitlab.(*gitlabImporter).ImportAll.func1(0xc000260100, 0xc0000aed00, 0xc0010bc000)
        /home/dtrckd/src/_sink/git-bug/bridge/gitlab/import.go:70 +0x547
created by github.com/MichaelMure/git-bug/bridge/gitlab.(*gitlabImporter).ImportAll
        /home/dtrckd/src/_sink/git-bug/bridge/gitlab/import.go:66 +0x146

Similar scenario on Github seems to work fine.

Version: git-bug version: 0.7.1-dev-28c3ee3b71 github.com/xanzy/go-gitlab v0.29.0 go version go1.14.1 linux/amd64

Gitlab: 12.9.2 (self hosted) OS: Linux 4.19.0-8-amd64 / Debian 10 x86_64

Adrien D. (dtrckd) commented (edited)

New test result: When using git-bug bridge by pushing bugs to remote before pulling, bug are exported. Then If we create an issue on Gitlab and import, the first time import is ok, then I got the crash (+ the doublon), and bridge push doesn't push new local bug anymore.

Michael Muré (MichaelMure) added label area/bridge

Michael Muré (MichaelMure) added label kind/bug

Glen Solsberry (gms8994) commented

I've received this error as well. Create an identity locally, then followed these steps:

  1. git bug bridge configure - set up gitlab as the bridge
git bug bridge push
new issue: e80238b17c133382dc01ba74eef316ad50d84e5c
exported 1 issues with gitlab bridge
  1. Made a change in the Gitlab UI (applied a label to the issue).
git bug bridge pull
new identity: 7cef8e713b2299d8a8035d038c42d5c29e75cda9
new issue: 0d4041eb258226f712c9391835e7faf409792dc9
imported 1 issues and 1 identities with gitlab bridge
  1. git bug termui - notice that there are now two issues listed, one with the label, one without.
git bug bridge pull
panic: runtime error: index out of range [-1]

goroutine 57 [running]:
github.com/MichaelMure/git-bug/bridge/gitlab/iterator.(*issueIterator).Value(...)
	/home/travis/gopath/src/github.com/MichaelMure/git-bug/bridge/gitlab/iterator/issue.go:38
github.com/MichaelMure/git-bug/bridge/gitlab/iterator.(*Iterator).NextIssue(0xc000296000, 0x0)
	/home/travis/gopath/src/github.com/MichaelMure/git-bug/bridge/gitlab/iterator/iterator.go:86 +0x244
github.com/MichaelMure/git-bug/bridge/gitlab.(*gitlabImporter).ImportAll.func1(0xc0002541c0, 0xc000142d80, 0xc00008c1e0)
	/home/travis/gopath/src/github.com/MichaelMure/git-bug/bridge/gitlab/import.go:70 +0x560
created by github.com/MichaelMure/git-bug/bridge/gitlab.(*gitlabImporter).ImportAll
	/home/travis/gopath/src/github.com/MichaelMure/git-bug/bridge/gitlab/import.go:66 +0x146

Performing a pull from within git bug termui "works," in that done is reported.

Glen Solsberry (gms8994) commented

This also seems to affect new instances of a gitlab bridge. After configuring in another repo, I attempted to git bug bridge pull and received the same panic as above.

Manfred Kröhnert (mkroehnert) commented

Hi, I am receiving the same panic: runtime error: index out of range [-1] on the repo that I used for testing in #360.

For reference:

  • I used git-bug 0.7.0 on https://gitlab.com/mkroehnert/git-bug-issue-test/-/issues (which had the label issue).
  • I ran git bug bridge pull with git-bug 0.7.1 on the same checkout that held the initial git bug bridge pull from the above mentioned ticket.
  • The exact error received after the bridge pull is (same as in the very first message):
panic: runtime error: index out of range [-1]

goroutine 23 [running]:
github.com/MichaelMure/git-bug/bridge/gitlab/iterator.(*issueIterator).Value(...)
        /build/source/bridge/gitlab/iterator/issue.go:38
github.com/MichaelMure/git-bug/bridge/gitlab/iterator.(*Iterator).NextIssue(0xc0000ae180, 0x0)
        /build/source/bridge/gitlab/iterator/iterator.go:86 +0x244
github.com/MichaelMure/git-bug/bridge/gitlab.(*gitlabImporter).ImportAll.func1(0xc0002161c0, 0xc0000aed00, 0xc00021c5a0)
        /build/source/bridge/gitlab/import.go:70 +0x547
created by github.com/MichaelMure/git-bug/bridge/gitlab.(*gitlabImporter).ImportAll
        /build/source/bridge/gitlab/import.go:66 +0x146

Michael Muré (MichaelMure) commented

Hey guys, sorry it's taking time. Life is busy at the moment...

Manfred Kröhnert (mkroehnert) commented

@mkroehnert take your time. I didn't post the example to pressure anyone but to contribute an example that hopefully helps in reproducing the issue.

Michael Muré (MichaelMure) commented

Btw, there is any steps done about this gitlab gridge?

Not at the moment. I'm pretty busy on other parts of the code, on top of my actual job. If someone could step up and debug this that would be very welcome.

Basxto (basxto) commented

Similar scenario on Github seems to work fine.

I did not get an error, but duplicated issues and comments happen for me on github too. All issues and comments created with git bug, they got duplicated after pulling what I pushed before. The exporter does not push the duplicates again.

Michael Muré (MichaelMure) closed the bug

Willow (stacyharper) commented

Thanks for the fix on the importer !

It seems there is still a bug on the exporter, somehow.

Pushing new bug or comment or edition (or I think anything) still give exported 0 issues with foobar bridge

Matthias Simon (5nord) commented (edited)

It seems there is still a bug on the exporter, somehow.

I also noticed two issues:

  • all imported bugs are "open" (when doing a git bug ls or git bug termui).
  • modified bugs are not exported.

But those issues seem not to be related with the original importer issues and I have not figured the root cause, yet.

Michael Muré (MichaelMure) opened the bug

Michael Muré (MichaelMure) commented

all imported bugs are "open" (when doing a git bug ls or git bug termui).

This is a known issue (although there is not one filed here). The problem is that the gitlab API changed and we don't have access to the history of state change anymore. See https://github.com/MichaelMure/git-bug/blob/master/bridge/gitlab/export_test.go#L245-L248

To be fair, it's not even a proper API. They call that "note", it's a sort of stream of untyped events. It was sort of enough at some point to get the full history of a bug but not anymore.

https://github.com/MichaelMure/git-bug/blob/master/bridge/gitlab/import_notes.go

One thing we could do is to create a status change in git-bug to match the final state, but that means we won't have the full history.

modified bugs are not exported.

No idea about this one.

Michael Muré (MichaelMure) commented

Well, that was 7 months ago, maybe the API changed again?

Matthias Simon (5nord) commented

Thank you for the context, Michael. I'll dig right into it. :-)

The GitLab issue API response seems to provide a state: {open, closed} field: https://docs.gitlab.com/ce/api/issues.html#single-issue. I could not find any references to a history, yet. However, I could imagine a single state without explicit history is sufficient for most use cases.

Michael Muré (MichaelMure) commented

However, I could imagine a single state without explicit history is sufficient for most use cases.

It would work (as in, you get the correct final state), but the history would be incorrect. That can be misleading and would definitely be seen as a bug.

Matthias Simon (5nord) commented

Maybe I am still missing a point. Are you talking about the issue history on gitlab side? Do git-bug bugs also have a history?

Michael Muré (MichaelMure) commented (edited)

git-bug has the full history of a bug (you can see it in the timeline: you can see, say, all the status change, not just the current state). Ideally, an importer would be able to import this full history to mirror exactly what happened on the remote tracker. The github importer does that, the Jira one as well, but gitlab doesn't provide a good enough API ... (I'll say it, it's quite crap).

Matthias Simon (5nord) commented (edited)

I understand the issues with the GitLab API.

I might have an idea how to fix the missing history in gitlab importer: Since GitLab 13.2 there's a API for retrieving issue state changes. When https://github.com/xanzy/go-gitlab/pull/1062 is accepted, I'd like to handle state changes similar to label changes.

If this also resolves any issue with exporting, I don't know, yet.

Michael Muré (MichaelMure) commented

@5nord that's awesome! Any idea if a modern Gitlab also include new API to replace the usage of other "notes" we use in the importer? It works now but it's also quite unstable ...

Matthias Simon (5nord) commented

It looks like it. However, I am not sure, because I am still familiarizing with git-bug and GitLab API.

Michael Muré (MichaelMure) commented

Closing as a lot have changed in the bridges. Please open a new issue if there is still problems.

Michael Muré (MichaelMure) closed the bug