Project

General

Profile

Enhancement #647

Patched softwares, how to handle them?

Added by Marc Dequènes 4 months ago. Updated 6 days ago.

Status:
In Progress
Priority:
Normal
Category:
-
Start date:
2019-04-21
Due date:
% Done:

10%

Estimated time:
Patch Available:
Confirmed:
No
Branch:
Entity:
DuckCorp
Security:
Help Needed:

Description

The following files were patched:
  • Toushirou:
    • /usr/share/redmine/app/controllers/activities_controller.rb: missing API to get activities on the website
    • /usr/share/redmine/lib/redmine/themes.rb: per-project theme
  • Jinta:
    • /usr/lib/python3/dist-packages/wikitrans/wikimarkup.py: fix for dico
  • Orfeo:
    • /usr/lib/ruby/vendor_ruby/mail/fields/content_type_field.rb: fix for cyborghood_postman

History

#1

Updated by Marc Dequènes 4 months ago

  • Parent task set to #648
#2

Updated by Marc Dequènes 4 months ago

For wikitrans, here is the diff:

@@ -1025,7 +1025,7 @@
                 self.file = open(keywords[kw])
             elif kw == 'text':
                 if sys.version_info[0] > 2:
-                    self.text = keywords[kw].decode('utf-8').split("\n")
+                    self.text = keywords[kw].split("\n")
                 else:
                     self.text = keywords[kw].split("\n")
             elif kw == 'lang':

Without it we get this trace:

May 08 20:10:45 Jinta dicod[25119]: Error: Traceback (most recent call last):
May 08 20:10:45 Jinta dicod[25119]: Error:   File "/usr/share/dico/python/mediawiki.py", line 98, in define_word
May 08 20:10:45 Jinta dicod[25119]: Error:     wikiparser = TextWiktionaryMarkup (text=data)
May 08 20:10:45 Jinta dicod[25119]: Error:   File "/usr/lib/python3/dist-packages/wikitrans/wiki2text.py", line 278, in __init__
May 08 20:10:45 Jinta dicod[25119]: Error:     super(TextWikiMarkup, self).__init__(*args, **keywords)
May 08 20:10:45 Jinta dicod[25119]: Error:   File "/usr/lib/python3/dist-packages/wikitrans/wikimarkup.py", line 1028, in __init__
May 08 20:10:45 Jinta dicod[25119]: Error:     self.text = keywords[kw].decode('utf-8').split("\n")
May 08 20:10:45 Jinta dicod[25119]: Error: AttributeError: 'str' object has no attribute 'decode'

It seems the decoding already happens before.

Interestingly if I use the wikitrans program with a WP URL the original code works but the fix breaks with:

# wikitrans   https://en.wikipedia.org/wiki/Special:Export/Coin
Traceback (most recent call last):
  File "/usr/bin/wikitrans", line 186, in <module>
    main()
  File "/usr/bin/wikitrans", line 175, in main
    markup = handlers[options.otype][options.itype](**options.kwdict)
  File "/usr/lib/python3/dist-packages/wikitrans/wiki2html.py", line 241, in __init__
    super(HtmlWikiMarkup, self).__init__(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/wikitrans/wikimarkup.py", line 1029, in __init__
    self.text = keywords[kw].split("\n")
TypeError: a bytes-like object is required, not 'str'

I looked into the dicod mediawiki plugin and stumbled onto my own patch for Python 3 :-).

I made this patch witch works with the original wikitrans code:

--- /usr/share/dico/python/mediawiki.py.orig    2019-05-08 20:45:22.746677470 +0200
+++ /usr/share/dico/python/mediawiki.py    2019-05-08 20:45:11.702138585 +0200
@@ -93,8 +93,7 @@
             data = el[0].firstChild.data
             if dico.current_markup () != 'wiki':
                 data = self.__htmlentitydecode (data)
-                if sys.version_info[0] == 2:
-                    data = data.encode ('utf-8')
+                data = data.encode ('utf-8')
                 wikiparser = TextWiktionaryMarkup (text=data)
                 wikiparser.parse ()
                 data = str (wikiparser)

So I believe either wikitrans should be more flexible, or this patch should be applied in dicod.

#3

Updated by Marc Dequènes 4 months ago

As for ruby-mail, here is the diff:

@@ -183,10 +183,8 @@
         params = params.map { |i| i.split(/\s*\=\s*/) }
         params = params.map { |i| "#{i[0]}=#{dquote(i[1].to_s.gsub(/;$/,""))}" }.join('; ')
         "#{type}; #{params}" 
-      when val =~ /^\s*$/
-        'text/plain'
       else
-        ''
+        'text/plain; charset=us-ascii'
       end
     end

@@ -195,7 +193,7 @@
       when val =~ /^([\w\-]+)\/([\w\-]+);.+$/i
         "#{$1}/#{$2}" 
       else
-        'text/plain'
+        'text/plain; charset=us-ascii'
       end
     end
   end

The version in Buster has slightly changed but the logic is equally broken and the patch is still needed.

#4

Updated by Marc Dequènes 4 months ago

  • Status changed from New to In Progress
  • Assignee set to Marc Dequènes
  • % Done changed from 0 to 10

As for Redmine changes, the patches are lost in Toushirou-OLD and we're trying to get them back. The changes also included a new file IIRC, as well as the Bip project theme.

Upstream did not make any progress on the topic so we'll have to port this work to the new version in Buster (see http://www.redmine.org/issues/2778).

#5

Updated by Marc Dequènes about 1 month ago

  • Parent task deleted (#648)
#6

Updated by Marc Dequènes about 1 month ago

#7

Updated by Marc Dequènes about 1 month ago

#8

Updated by Marc Dequènes about 1 month ago

#9

Updated by Marc Dequènes 6 days ago

ruby-mail is blocked at 2.6.4 to avoid breaking Banya (which is probably broken anyway). I need to evaluate 2.7.1 if we decide to keep it.

#10

Updated by Marc Dequènes 5 days ago

Also available in: Atom PDF