Github bridge import: error, probably caused by line of 65536+ length

Labels: kind/bug

Timeline

rng-dynamics (rng-dynamics) opened

I tried to import all the issues from https://github.com/golang/go. (Currently I am using git-bug from PR #585).

This gives the following error:

import error: issue creation: key too large
imported 18123 issues and 5456 identities with default bridge

The error is caused when importing https://github.com/golang/go/issues/18201 . This issue contains a line of 65536+ length.

rng-dynamics (rng-dynamics) added label kind/bug

Michael Muré (MichaelMure) commented

Hu, that's a funny one.

Are you able to pinpoint a bit more where this error happen?

rng-dynamics (rng-dynamics) commented (edited)

At the moment I do not know where the error happens exactly. Just looking at the error message it should originiate from somewhere below the call to repo.NewBugRaw().

The cause might be the same as described in the issue which causes the error. There the cause was a bufio.Scanner (which has bufio.MaxScanTokenSize equal to 65536). The solution which the golang people found was to replace the bufio.Scanner by a bufio.Reader as described in https://go-review.googlesource.com/c/tools/+/93439/

Michael Muré (MichaelMure) commented

So, what's happening is that this text is passed to Bleve (full text search) for indexing, Bleve process it then try to write a piece of data (with a very long key) to BoltDB (a key/value store).

I filled an issue here: https://github.com/blevesearch/bleve/issues/1576

Michael Muré (MichaelMure) closed the bug