Refine syntax highlighting for Python docstrings (#20898)

jfmontanaro and Piotr Osiewicz created 11 months ago

Following up on #20763, this PR adds support for module- and class-level
docstrings, adds "additional docstrings" as described in [PEP
257](https://peps.python.org/pep-0257/), and fixes function-level
docstrings so that only the first string literal in a function gets
treated as a docstring.

One question that occurs to me is: Would it be good to capture attribute
and additional docstrings differently from regular docstrings? E.g.
`@string.doc.attribute`, `@string.doc.additional`? PEP 257 mentions that
unlike regular docstrings, these docstrings are ignored by the
interpreter (regular docstrings get added as the `__doc__` property of
the object they document), so I can see someone potentially wanting to
style them a little differently.

Release notes:

* Added Python syntax highlighting for class- and module-level
docstrings, additional docstrings, and improved recognition of
function-level docstrings.

Co-authored-by: Piotr Osiewicz <[e-mail]>

Change summary

crates/languages/src/python/highlights.scm | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)

Detailed changes

crates/languages/src/python/highlights.scm

@@ -128,33 +128,39 @@
   "}" @punctuation.special) @embedded
 
 ; Docstrings.
+(module
+  .(expression_statement (string) @string.doc)+)
+
+(class_definition
+  body: (block .(expression_statement (string) @string.doc)+))
+
 (function_definition
   "async"?
   "def"
   name: (_)
   (parameters)?
-  body: (block . (expression_statement (string) @string.doc)))
+  body: (block .(expression_statement (string) @string.doc)+))
 
 (class_definition
   body: (block
     . (comment) @comment*
-    . (expression_statement (string) @string.doc)))
+    . (expression_statement (string) @string.doc)+))
 
 (module
   . (comment) @comment*
-  . (expression_statement (string) @string.doc))
+  . (expression_statement (string) @string.doc)+)
 
 (module
   [
     (expression_statement (assignment))
     (type_alias_statement)
   ]
-  . (expression_statement (string) @string.doc))
+  . (expression_statement (string) @string.doc)+)
 
 (class_definition
   body: (block
     (expression_statement (assignment))
-    . (expression_statement (string) @string.doc)))
+    . (expression_statement (string) @string.doc)+))
 
 (class_definition
   body: (block
@@ -163,7 +169,7 @@
       (#eq? @function.method.constructor "__init__")
       body: (block
         (expression_statement (assignment))
-        . (expression_statement (string) @string.doc)))))
+        . (expression_statement (string) @string.doc)+))))
 
 
 [