1# JIRA Bridge
2
3## Design Notes
4
5### One bridge = one project
6
7There aren't any huge technical barriers requiring this, but since git-bug lacks
8a notion of "project" there is no way to know which project to export new bugs
9to as issues. Also, JIRA projects are first-class immutable metadata and so we
10*must* get it right on export. Therefore the bridge is configured with the `Key`
11for the project it is assigned to. It will only import bugs from that project.
12
13### JIRA fields
14
15The bridge currently does nothing to import any of the JIRA fields that don't
16have `git-bug` equivalents ("Assignee", "sprint", "story points", etc).
17Hopefully the bridge will be able to enable synchronization of these soon.
18
19### Credentials
20
21JIRA does not support user/personal access tokens. They have experimental
223-legged oauth support but that requires an API token for the app configured
23by the server administrator. The only reliable authentication mechanism then is
24the username/password and session-token mechanims. We can aquire a session
25token programatically from the username/password but these are very short lived
26(i.e. hours or less). As such the bridge currently requires an actual username
27and password as user credentials. It supports three options:
28
291. Storing both username and password in a separate file referred to by
30 the `git-config` (I like to use `.git/jira-credentials.json`)
312. Storing the username and password in clear-text in the git config
323. Storing the username only in the git config and asking for the password
33 on each `push` or `pull`.
34
35### Issue Creation Defaults
36
37When a new issues is created in JIRA there are often certain mandatory fields
38that require a value or the creation is rejected. In the issue create form on
39the JIRA web interface, these are annotated as "required". The `issuetype` is
40always required (e.g. "bug", "story", "task", etc). The set of required metadata
41is configurable (in JIRA) per `issuetype` so the set might be different between
42"bug" and "story", for example.
43
44For now, the bridge only supports exporting issues as a single `issuetype`. If
45no configuration is provied, then the default is `"id": "10001"` which is
46`"story"` in the default set of issue types.
47
48In addition to specifying the `issuetype` of issues created on export, the
49bridge will also allow you to specify a constant global set of default values
50for any additional required fields. See the configuration section below for the
51syntax.
52
53For longer term goals, see the section below on workflow validation
54
55### Assign git-bug id to field during issue creation
56
57JIRA allows for the inclusion of custom "fields" in all of their issues. The
58JIRA bridge will store the JIRA issue "id" for any bugs which are synchronized
59to JIRA, but it can also assign to a custom JIRA `field` the `git-bug` id. This
60way the `git-bug` id can be displayed in the JIRA web interface and certain
61integration activities become easier.
62
63See the configuration section below on how to specify the custom field where the
64JIRA bridge should write this information.
65
66
67### Workflows and Transitions
68
69JIRA issue states are subject to customizable "workflows" (project managers
70apparently validate themselves by introducing developer friction). In general,
71issues can only transition from one state to another if there is an edge between
72them in the state graph (a.k.a. "workflow"). JIRA calls these edges
73"transitions". Furthermore, each transition may include a set of mandatory
74fields which must be set in order for the transition to succeed. For example the
75transition of `"status"` from `"In Progress"` to `"Closed"` might required a
76`"resolution"` (i.e. `"Fixed"` or `"Working as intended"`).
77
78Dealing with complex workflows is going to be challenging. Some long-term
79aspirations are described in the section below on "Workflow Validation".
80Currently the JIRA bridge isn't very smart about transitions though, so you'll
81need to tell it what you want it to do when importing and exporting a state
82change (i.e. to "close" or "open" a bug). Currently the bridge accepts
83configuration options which map the two `git-bug` statuses ("open", "closed") to
84two JIRA statuses. On import, the JIRA status is mapped to a `git-bug` status
85(if a mapping exists) and the `git-bug` status is assigned. On export, the
86`git-bug` status is mapped to a JIRA status and if a mapping exists the bridge
87will query the list of available transitions for the issue. If a transition
88exists to the desired state the bridge will attempt to execute the transition.
89It does not currently support assigning any fields during the transition so if
90any fields are required the transition will fail during export and the status
91will be out of sync.
92
93### JIRA Changelog
94
95Some operations on JIRA issues are visible in a timeline view known as the
96`changelog`. The JIRA cloud product provides an
97`/issue/{issueIdOrKey}/changelog` endpoint which provides a paginated view but
98the JIRA server product does not. The changelog is visible by querying the issue
99with the `expand=changelog` query parameter. Unfortunately in this case the
100entire changelog is provided without paging.
101
102Each changelog entry is identified with a unique string `id`, but within a
103single changelog entry is a list of multilple fields that are modified. In other
104words a single "event" might atomically change multiple fields. As an example,
105when an issue is closed the `"status"` might change to `"closed"` and the
106`"resolution"` might change to `"fixed'`.
107
108When a changelog entry is imported by the JIRA bridge, each individual field
109that was changed is treated as a separate `git-bug` operation. In other words a
110single JIRA change event might create more than one `git-bug` operation.
111
112However, when a `git-bug` operation is exported to JIRA it will only create a
113single changelog entry. Furthermore, when we modify JIRA issues over the REST
114API JIRA does not provide any information to associate that modification event
115with the changelog. We must, therefore, herustically match changelog entries
116against operations that we performed in order to not import them as duplicate
117events. In order to assist in this matching proceess, the bridge will record the
118JIRA server time of the response to the `POST` (as reported by the `"Date"`
119response header). During import, we keep an iterator to the list of `git-bug`
120operations for the bug mapped to the Jira issue. As we walk the JIRA changelog,
121we keep the iterator pointing to the first operation with an annotation which is
122*not before* that changelog entry. If the changelog entry is the result of an
123exported `git-bug` operation, then this must be that operation. We then scan
124through the list of changeitems (changed fields) in the changelog entry, and if
125we can match a changed field to the candidate `git-bug` operation then we have
126identified the match.
127
128### Unlogged Changes
129
130Comments (creation and edition) do not show up in the JIRA changelog. However
131JIRA reports both a `created` and `updated` date for each comment. If we
132import a comment which has an `updated` and `created` field which do not match,
133then we treat that as a new comment edition. If we do not already have the
134comment imported, then we import an empty comment followed by a comment edition.
135
136Because comment editions are not uniquely identified in JIRA we identify them
137in `git-bug` by concatinating the JIRA issue `id` with the `updated` time of
138the edition.
139
140### Workflow Validation (future)
141
142The long-term plan for the JIRA bridge is to download and store the workflow
143specifiations from the JIRA server. This includes the required metadata for
144issue creation, and the status state graph, and the set of required metadata for
145status transition.
146
147When an existing `git-bug` is initially marked for export, the bridge will hook
148in and validate the bug state against the required metadata. Then it will prompt
149for any missing metadata using a set of UI components appropriate for the field
150schema as reported by JIRA. If the user cancels then the bug will not be marked
151for export.
152
153When a bug already marked for JIRA export (including those that were imported)
154is modified, the bridge will hook in and validate the modification against the
155workflow specifications. It will prompt for any missing metadata as in the
156creation process.
157
158During export, the bridge will validate any export operations and skip them if
159we know they will fail due to violation of the cached workflow specification
160(i.e. missing required fields for a transition). A list of bugs "blocked for
161export" will be available to query. A UI command will allow the user to inspect
162and resolve any bugs that are "blocked for export".
163
164## Configuration
165
166As mentioned in the notes above, there are a few optional configuration fields
167that can be set beyond those that are prompted for during the initial bridge
168configuration. You can set these options in your `.git/config` file:
169
170### Issue Creation Defaults
171
172The format for this config entry is a JSON object containing fields you wish to
173set during issue creation when exproting bugs. If you provide a value for this
174configuration option, it must include at least the `"issuetype"` field, or
175the bridge will not be able to export any new issues.
176
177Let's say that we want bugs exported to JIRA to have a default issue type of
178"Story" which is `issuetype` with id `10001`. Then we will add the following
179entry to our git-config:
180
181```
182create-issue-defaults = {"issuetype":"10001"}
183```
184
185If you needed an additional required field `customfield_1234` and you wanted to
186provide a default value of `"default"` then you would add the following to your
187config:
188
189```
190create-issue-defaults = {"issuetype":"10001","customfield_1234":"default"}
191```
192
193Note that the content of this value is merged verbatim to the JSON object that
194is `POST`ed to the JIRA rest API, so you can use arbitrary valid JSON.
195
196
197### Assign git-bug id to field
198
199If you want the bridge to fill a JIRA field with the `git-bug` id when exporting
200issues, then provide the name of the field:
201
202```
203create-issue-gitbug-id = "customfield_5678"
204```
205
206### Status Map
207
208You can specify the mapping between `git-bug` status and JIRA status id's using
209the following:
210```
211bug-open-id = 1
212bug-closed-id = 6
213```
214
215Note that in JIRA each different `issuetype` can have a different set of
216statuses. The bridge doesn't currently support more than one mapping, however.
217
218### Full example
219
220Here is an example configuration with all optional fields set
221```
222[git-bug "bridge.default"]
223 project = PROJ
224 credentials-file = .git/jira-credentials.json
225 target = jira
226 server = https://jira.example.com
227 create-issue-defaults = {"issuetype":"10001","customfield_1234":"default"}
228 create-issue-gitbug-id = "customfield_5678"
229 bug-open-id = 1
230 bug-closed-id = 6
231```
232
233## To-Do list
234
235* [0cf5c71] Assign git-bug to jira field on import
236* [8acce9c] Download and cache workflow representation
237* [95e3d45] Implement workflow gui
238* [c70e22a] Implement additional query filters for import
239* [9ecefaa] Create JIRA mock and add REST unit tests
240* [67bf520] Create import/export integration tests
241* [1121826] Add unit tests for utilites
242* [0597088] Use OS keyring for credentials
243* [d3e8f79] Don't count on the `Total` value in paginations
244
245
246## Using CURL to poke at your JIRA's REST API
247
248If you need to lookup the `id` for any `status`es or the `schema` for any
249creation metadata, you can use CURL to query the API from the command line.
250Here are a couple of examples to get you started.
251
252### Getting a session token
253
254```
255curl \
256 --data '{"username":"<username>", "password":"<password>"}' \
257 --header "Content-Type: application/json" \
258 --request POST \
259 <serverUrl>/rest/auth/1/session
260```
261
262**Note**: If you have a json pretty printer installed (`sudo apt install jq`),
263pipe the output through through that to make things more readable:
264
265```
266curl --silent \
267 --data '{"username":"<username>", "password":"<password>"}' \
268 --header "Content-Type: application/json" \
269 --request POST
270 <serverUrl>/rest/auth/1/session | jq .
271```
272
273example output:
274```
275{
276 "session": {
277 "name": "JSESSIONID",
278 "value": "{sessionToken}"
279 },
280 "loginInfo": {
281 "loginCount": 268,
282 "previousLoginTime": "2019-11-12T08:03:35.300-0800"
283 }
284}
285```
286
287Make note of the output value. On subsequent invocations of `curl`, append the
288following command-line option:
289
290```
291--cookie "JSESSIONID={sessionToken}"
292```
293
294Where `{sessionToken}` is the output from the `POST` above.
295
296### Get a list of issuetype ids
297
298```
299curl --silent \
300 --cookie "JSESSIONID={sessionToken}" \
301 --header "Content-Type: application/json" \
302 --request GET https://jira.example.com/rest/api/2/issuetype \
303 | jq .
304```
305
306**example output**:
307```
308 {
309 "self": "https://jira.example.com/rest/api/2/issuetype/13105",
310 "id": "13105",
311 "description": "",
312 "iconUrl": "https://jira.example.com/secure/viewavatar?size=xsmall&avatarId=10316&avatarType=issuetype",
313 "name": "Test Plan Links",
314 "subtask": true,
315 "avatarId": 10316
316 },
317 {
318 "self": "https://jira.example.com/rest/api/2/issuetype/13106",
319 "id": "13106",
320 "description": "",
321 "iconUrl": "https://jira.example.com/secure/viewavatar?size=xsmall&avatarId=10316&avatarType=issuetype",
322 "name": "Enable Initiatives on the project",
323 "subtask": true,
324 "avatarId": 10316
325 },
326 ...
327```
328
329
330### Get a list of statuses
331
332
333```
334curl --silent \
335 --cookie "JSESSIONID={sessionToken}" \
336 --header "Content-Type: application/json" \
337 --request GET https://jira.example.com/rest/api/2/project/{projectIdOrKey}/statuses \
338 | jq .
339```
340
341**example output:**
342```
343[
344 {
345 "self": "https://example.com/rest/api/2/issuetype/3",
346 "id": "3",
347 "name": "Task",
348 "subtask": false,
349 "statuses": [
350 {
351 "self": "https://example.com/rest/api/2/status/1",
352 "description": "The issue is open and ready for the assignee to start work on it.",
353 "iconUrl": "https://example.com/images/icons/statuses/open.png",
354 "name": "Open",
355 "id": "1",
356 "statusCategory": {
357 "self": "https://example.com/rest/api/2/statuscategory/2",
358 "id": 2,
359 "key": "new",
360 "colorName": "blue-gray",
361 "name": "To Do"
362 }
363 },
364...
365```