1# JIRA Bridge
2
3## Design Notes
4
5### One bridge = one project
6
7There aren't any huge technical barriers requiring this, but since git-bug lacks
8a notion of "project" there is no way to know which project to export new bugs
9to as issues. Also, JIRA projects are first-class immutable metadata and so we
10*must* get it right on export. Therefore the bridge is configured with the `Key`
11for the project it is assigned to. It will only import bugs from that project.
12
13### JIRA fields
14
15The bridge currently does nothing to import any of the JIRA fields that don't
16have `git-bug` equivalents ("Assignee", "sprint", "story points", etc).
17Hopefully the bridge will be able to enable synchronization of these soon.
18
19### Credentials
20
21JIRA does not support user/personal access tokens. They have experimental
223-legged oauth support but that requires an API token for the app configured by
23the server administrator. The only reliable authentication mechanism then is the
24username/password and session-token mechanism. We can acquire a session token
25programmatically from the username/password but these are very short lived (i.e.
26hours or less). As such the bridge currently requires an actual username and
27password as user credentials. It supports three options:
28
291. Storing both username and password in a separate file referred to by the
30 `git-config` (I like to use `.git/jira-credentials.json`)
312. Storing the username and password in clear-text in the git config
323. Storing the username only in the git config and asking for the password on
33 each `push` or `pull`.
34
35### Issue Creation Defaults
36
37When a new issues is created in JIRA there are often certain mandatory fields
38that require a value or the creation is rejected. In the issue create form on
39the JIRA web interface, these are annotated as "required". The `issuetype` is
40always required (e.g. "bug", "story", "task", etc). The set of required metadata
41is configurable (in JIRA) per `issuetype` so the set might be different between
42"bug" and "story", for example.
43
44For now, the bridge only supports exporting issues as a single `issuetype`. If
45no configuration is provided, then the default is `"id": "10001"` which is
46`"story"` in the default set of issue types.
47
48In addition to specifying the `issuetype` of issues created on export, the
49bridge will also allow you to specify a constant global set of default values
50for any additional required fields. See the configuration section below for the
51syntax.
52
53For longer term goals, see the section below on workflow validation
54
55### Assign git-bug id to field during issue creation
56
57JIRA allows for the inclusion of custom "fields" in all of their issues. The
58JIRA bridge will store the JIRA issue "id" for any bugs which are synchronized
59to JIRA, but it can also assign to a custom JIRA `field` the `git-bug` id. This
60way the `git-bug` id can be displayed in the JIRA web interface and certain
61integration activities become easier.
62
63See the configuration section below on how to specify the custom field where the
64JIRA bridge should write this information.
65
66### Workflows and Transitions
67
68JIRA issue states are subject to customizable "workflows" (project managers
69apparently validate themselves by introducing developer friction). In general,
70issues can only transition from one state to another if there is an edge between
71them in the state graph (a.k.a. "workflow"). JIRA calls these edges
72"transitions". Furthermore, each transition may include a set of mandatory
73fields which must be set in order for the transition to succeed. For example the
74transition of `"status"` from `"In Progress"` to `"Closed"` might required a
75`"resolution"` (i.e. `"Fixed"` or `"Working as intended"`).
76
77Dealing with complex workflows is going to be challenging. Some long-term
78aspirations are described in the section below on "Workflow Validation".
79Currently the JIRA bridge isn't very smart about transitions though, so you'll
80need to tell it what you want it to do when importing and exporting a state
81change (i.e. to "close" or "open" a bug). Currently the bridge accepts
82configuration options which map the two `git-bug` statuses ("open", "closed") to
83two JIRA statuses. On import, the JIRA status is mapped to a `git-bug` status
84(if a mapping exists) and the `git-bug` status is assigned. On export, the
85`git-bug` status is mapped to a JIRA status and if a mapping exists the bridge
86will query the list of available transitions for the issue. If a transition
87exists to the desired state the bridge will attempt to execute the transition.
88It does not currently support assigning any fields during the transition so if
89any fields are required the transition will fail during export and the status
90will be out of sync.
91
92### JIRA Changelog
93
94Some operations on JIRA issues are visible in a timeline view known as the
95`changelog`. The JIRA cloud product provides an
96`/issue/{issueIdOrKey}/changelog` endpoint which provides a paginated view but
97the JIRA server product does not. The changelog is visible by querying the issue
98with the `expand=changelog` query parameter. Unfortunately in this case the
99entire changelog is provided without paging.
100
101Each changelog entry is identified with a unique string `id`, but within a
102single changelog entry is a list of multiple fields that are modified. In other
103words a single "event" might atomically change multiple fields. As an example,
104when an issue is closed the `"status"` might change to `"closed"` and the
105`"resolution"` might change to `"fixed'`.
106
107When a changelog entry is imported by the JIRA bridge, each individual field
108that was changed is treated as a separate `git-bug` operation. In other words a
109single JIRA change event might create more than one `git-bug` operation.
110
111However, when a `git-bug` operation is exported to JIRA it will only create a
112single changelog entry. Furthermore, when we modify JIRA issues over the REST
113API JIRA does not provide any information to associate that modification event
114with the changelog. We must, therefore, heuristically match changelog entries
115against operations that we performed in order to not import them as duplicate
116events. In order to assist in this matching process, the bridge will record the
117JIRA server time of the response to the `POST` (as reported by the `"Date"`
118response header). During import, we keep an iterator to the list of `git-bug`
119operations for the bug mapped to the Jira issue. As we walk the JIRA changelog,
120we keep the iterator pointing to the first operation with an annotation which is
121*not before* that changelog entry. If the changelog entry is the result of an
122exported `git-bug` operation, then this must be that operation. We then scan
123through the list of changeitems (changed fields) in the changelog entry, and if
124we can match a changed field to the candidate `git-bug` operation then we have
125identified the match.
126
127### Unlogged Changes
128
129Comments (creation and edition) do not show up in the JIRA changelog. However
130JIRA reports both a `created` and `updated` date for each comment. If we import
131a comment which has an `updated` and `created` field which do not match, then we
132treat that as a new comment edition. If we do not already have the comment
133imported, then we import an empty comment followed by a comment edition.
134
135Because comment editions are not uniquely identified in JIRA we identify them in
136`git-bug` by concatenating the JIRA issue `id` with the `updated` time of the
137edition.
138
139### Workflow Validation (future)
140
141The long-term plan for the JIRA bridge is to download and store the workflow
142specifications from the JIRA server. This includes the required metadata for
143issue creation, and the status state graph, and the set of required metadata for
144status transition.
145
146When an existing `git-bug` is initially marked for export, the bridge will hook
147in and validate the bug state against the required metadata. Then it will prompt
148for any missing metadata using a set of UI components appropriate for the field
149schema as reported by JIRA. If the user cancels then the bug will not be marked
150for export.
151
152When a bug already marked for JIRA export (including those that were imported)
153is modified, the bridge will hook in and validate the modification against the
154workflow specifications. It will prompt for any missing metadata as in the
155creation process.
156
157During export, the bridge will validate any export operations and skip them if
158we know they will fail due to violation of the cached workflow specification
159(i.e. missing required fields for a transition). A list of bugs "blocked for
160export" will be available to query. A UI command will allow the user to inspect
161and resolve any bugs that are "blocked for export".
162
163## Configuration
164
165As mentioned in the notes above, there are a few optional configuration fields
166that can be set beyond those that are prompted for during the initial bridge
167configuration. You can set these options in your `.git/config` file:
168
169### Issue Creation Defaults
170
171The format for this config entry is a JSON object containing fields you wish to
172set during issue creation when exporting bugs. If you provide a value for this
173configuration option, it must include at least the `"issuetype"` field, or the
174bridge will not be able to export any new issues.
175
176Let's say that we want bugs exported to JIRA to have a default issue type of
177"Story" which is `issuetype` with id `10001`. Then we will add the following
178entry to our git-config:
179
180```
181create-issue-defaults = {"issuetype":"10001"}
182```
183
184If you needed an additional required field `customfield_1234` and you wanted to
185provide a default value of `"default"` then you would add the following to your
186config:
187
188```
189create-issue-defaults = {"issuetype":"10001","customfield_1234":"default"}
190```
191
192Note that the content of this value is merged verbatim to the JSON object that
193is `POST`ed to the JIRA rest API, so you can use arbitrary valid JSON.
194
195### Assign git-bug id to field
196
197If you want the bridge to fill a JIRA field with the `git-bug` id when exporting
198issues, then provide the name of the field:
199
200```
201create-issue-gitbug-id = "customfield_5678"
202```
203
204### Status Map
205
206You can specify the mapping between `git-bug` status and JIRA status id's using
207the following:
208
209```
210bug-id-map = {\"open\": \"1\", \"closed\": \"6\"}
211```
212
213The format of the map is `<git-bug-status-name>: <jira-status-id>`. In general
214your jira instance will have more statuses than `git-bug` will and you may map
215more than one jira-status to a git-bug status. You can do this with
216`bug-id-revmap`:
217
218```
219bug-id-revmap = {\"10109\": \"open\", \"10006\": \"open\", \"10814\": \"open\"}
220```
221
222The reverse map `bug-id-revmap` will automatically include the inverse of the
223forward map `bug-id-map`.
224
225Note that in JIRA each different `issuetype` can have a different set of
226statuses. The bridge doesn't currently support more than one mapping, however.
227Also, note that the format of the map is JSON and the git config file syntax
228requires doublequotes to be escaped (as in the examples above).
229
230### Full example
231
232Here is an example configuration with all optional fields set
233
234```
235[git-bug "bridge.default"]
236 project = PROJ
237 credentials-file = .git/jira-credentials.json
238 target = jira
239 server = https://jira.example.com
240 create-issue-defaults = {"issuetype":"10001","customfield_1234":"default"}
241 create-issue-gitbug-id = "customfield_5678"
242 bug-open-id = 1
243 bug-closed-id = 6
244```
245
246## To-Do list
247
248- \[0cf5c71\] Assign git-bug to jira field on import
249- \[8acce9c\] Download and cache workflow representation
250- \[95e3d45\] Implement workflow gui
251- \[c70e22a\] Implement additional query filters for import
252- \[9ecefaa\] Create JIRA mock and add REST unit tests
253- \[67bf520\] Create import/export integration tests
254- \[1121826\] Add unit tests for utilities
255- \[0597088\] Use OS keyring for credentials
256- \[d3e8f79\] Don't count on the `Total` value in paginations
257
258## Using CURL to poke at your JIRA's REST API
259
260If you need to lookup the `id` for any `status`es or the `schema` for any
261creation metadata, you can use CURL to query the API from the command line. Here
262are a couple of examples to get you started.
263
264### Getting a session token
265
266```
267curl \
268 --data '{"username":"<username>", "password":"<password>"}' \
269 --header "Content-Type: application/json" \
270 --request POST \
271 <serverUrl>/rest/auth/1/session
272```
273
274[!NOTE]
275If you have a json pretty printer installed (`sudo apt install jq`), pipe the
276output through through that to make things more readable:
277
278```
279curl --silent \
280 --data '{"username":"<username>", "password":"<password>"}' \
281 --header "Content-Type: application/json" \
282 --request POST
283 <serverUrl>/rest/auth/1/session | jq .
284```
285
286example output:
287
288```
289{
290 "session": {
291 "name": "JSESSIONID",
292 "value": "{sessionToken}"
293 },
294 "loginInfo": {
295 "loginCount": 268,
296 "previousLoginTime": "2019-11-12T08:03:35.300-0800"
297 }
298}
299```
300
301Make note of the output value. On subsequent invocations of `curl`, append the
302following command-line option:
303
304```
305--cookie "JSESSIONID={sessionToken}"
306```
307
308Where `{sessionToken}` is the output from the `POST` above.
309
310### Get a list of issuetype ids
311
312```
313curl --silent \
314 --cookie "JSESSIONID={sessionToken}" \
315 --header "Content-Type: application/json" \
316 --request GET https://jira.example.com/rest/api/2/issuetype \
317 | jq .
318```
319
320**example output**:
321
322```
323 {
324 "self": "https://jira.example.com/rest/api/2/issuetype/13105",
325 "id": "13105",
326 "description": "",
327 "iconUrl": "https://jira.example.com/secure/viewavatar?size=xsmall&avatarId=10316&avatarType=issuetype",
328 "name": "Test Plan Links",
329 "subtask": true,
330 "avatarId": 10316
331 },
332 {
333 "self": "https://jira.example.com/rest/api/2/issuetype/13106",
334 "id": "13106",
335 "description": "",
336 "iconUrl": "https://jira.example.com/secure/viewavatar?size=xsmall&avatarId=10316&avatarType=issuetype",
337 "name": "Enable Initiatives on the project",
338 "subtask": true,
339 "avatarId": 10316
340 },
341 ...
342```
343
344### Get a list of statuses
345
346```
347curl --silent \
348 --cookie "JSESSIONID={sessionToken}" \
349 --header "Content-Type: application/json" \
350 --request GET https://jira.example.com/rest/api/2/project/{projectIdOrKey}/statuses \
351 | jq .
352```
353
354**example output:**
355
356```
357[
358 {
359 "self": "https://example.com/rest/api/2/issuetype/3",
360 "id": "3",
361 "name": "Task",
362 "subtask": false,
363 "statuses": [
364 {
365 "self": "https://example.com/rest/api/2/status/1",
366 "description": "The issue is open and ready for the assignee to start work on it.",
367 "iconUrl": "https://example.com/images/icons/statuses/open.png",
368 "name": "Open",
369 "id": "1",
370 "statusCategory": {
371 "self": "https://example.com/rest/api/2/statuscategory/2",
372 "id": 2,
373 "key": "new",
374 "colorName": "blue-gray",
375 "name": "To Do"
376 }
377 },
378...
379```