Project

General

Profile

Enhancement #647

Patched softwares, how to handle them?

Added by Marc Dequènes 6 months ago. Updated about 2 months ago.

Status:
In Progress
Priority:
Normal
Category:
-
Start date:
2019-04-21
Due date:
% Done:

10%

Estimated time:
Patch Available:
Confirmed:
No
Branch:
Entity:
DuckCorp
Security:
Help Needed:

Description

The following files were patched:
  • Toushirou:
    • /usr/share/redmine/app/controllers/activities_controller.rb: missing API to get activities on the website
    • /usr/share/redmine/lib/redmine/themes.rb: per-project theme
  • Jinta:
    • /usr/lib/python3/dist-packages/wikitrans/wikimarkup.py: fix for dico
  • Orfeo:
    • /usr/lib/ruby/vendor_ruby/mail/fields/content_type_field.rb: fix for cyborghood_postman

History

#1

Updated by Marc Dequènes 6 months ago

  • Parent task set to #648
#2

Updated by Marc Dequènes 6 months ago

For wikitrans, here is the diff:

@@ -1025,7 +1025,7 @@
                 self.file = open(keywords[kw])
             elif kw == 'text':
                 if sys.version_info[0] > 2:
-                    self.text = keywords[kw].decode('utf-8').split("\n")
+                    self.text = keywords[kw].split("\n")
                 else:
                     self.text = keywords[kw].split("\n")
             elif kw == 'lang':

Without it we get this trace:

May 08 20:10:45 Jinta dicod[25119]: Error: Traceback (most recent call last):
May 08 20:10:45 Jinta dicod[25119]: Error:   File "/usr/share/dico/python/mediawiki.py", line 98, in define_word
May 08 20:10:45 Jinta dicod[25119]: Error:     wikiparser = TextWiktionaryMarkup (text=data)
May 08 20:10:45 Jinta dicod[25119]: Error:   File "/usr/lib/python3/dist-packages/wikitrans/wiki2text.py", line 278, in __init__
May 08 20:10:45 Jinta dicod[25119]: Error:     super(TextWikiMarkup, self).__init__(*args, **keywords)
May 08 20:10:45 Jinta dicod[25119]: Error:   File "/usr/lib/python3/dist-packages/wikitrans/wikimarkup.py", line 1028, in __init__
May 08 20:10:45 Jinta dicod[25119]: Error:     self.text = keywords[kw].decode('utf-8').split("\n")
May 08 20:10:45 Jinta dicod[25119]: Error: AttributeError: 'str' object has no attribute 'decode'

It seems the decoding already happens before.

Interestingly if I use the wikitrans program with a WP URL the original code works but the fix breaks with:

# wikitrans   https://en.wikipedia.org/wiki/Special:Export/Coin
Traceback (most recent call last):
  File "/usr/bin/wikitrans", line 186, in <module>
    main()
  File "/usr/bin/wikitrans", line 175, in main
    markup = handlers[options.otype][options.itype](**options.kwdict)
  File "/usr/lib/python3/dist-packages/wikitrans/wiki2html.py", line 241, in __init__
    super(HtmlWikiMarkup, self).__init__(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/wikitrans/wikimarkup.py", line 1029, in __init__
    self.text = keywords[kw].split("\n")
TypeError: a bytes-like object is required, not 'str'

I looked into the dicod mediawiki plugin and stumbled onto my own patch for Python 3 :-).

I made this patch witch works with the original wikitrans code:

--- /usr/share/dico/python/mediawiki.py.orig    2019-05-08 20:45:22.746677470 +0200
+++ /usr/share/dico/python/mediawiki.py    2019-05-08 20:45:11.702138585 +0200
@@ -93,8 +93,7 @@
             data = el[0].firstChild.data
             if dico.current_markup () != 'wiki':
                 data = self.__htmlentitydecode (data)
-                if sys.version_info[0] == 2:
-                    data = data.encode ('utf-8')
+                data = data.encode ('utf-8')
                 wikiparser = TextWiktionaryMarkup (text=data)
                 wikiparser.parse ()
                 data = str (wikiparser)

So I believe either wikitrans should be more flexible, or this patch should be applied in dicod.

#3

Updated by Marc Dequènes 6 months ago

As for ruby-mail, here is the diff:

@@ -183,10 +183,8 @@
         params = params.map { |i| i.split(/\s*\=\s*/) }
         params = params.map { |i| "#{i[0]}=#{dquote(i[1].to_s.gsub(/;$/,""))}" }.join('; ')
         "#{type}; #{params}" 
-      when val =~ /^\s*$/
-        'text/plain'
       else
-        ''
+        'text/plain; charset=us-ascii'
       end
     end

@@ -195,7 +193,7 @@
       when val =~ /^([\w\-]+)\/([\w\-]+);.+$/i
         "#{$1}/#{$2}" 
       else
-        'text/plain'
+        'text/plain; charset=us-ascii'
       end
     end
   end

The version in Buster has slightly changed but the logic is equally broken and the patch is still needed.

#4

Updated by Marc Dequènes 6 months ago

  • Status changed from New to In Progress
  • Assignee set to Marc Dequènes
  • % Done changed from 0 to 10

As for Redmine changes, the patches are lost in Toushirou-OLD and we're trying to get them back. The changes also included a new file IIRC, as well as the Bip project theme.

Upstream did not make any progress on the topic so we'll have to port this work to the new version in Buster (see http://www.redmine.org/issues/2778).

#5

Updated by Marc Dequènes 4 months ago

  • Parent task deleted (#648)
#6

Updated by Marc Dequènes 4 months ago

#7

Updated by Marc Dequènes 4 months ago

#8

Updated by Marc Dequènes 4 months ago

#9

Updated by Marc Dequènes 2 months ago

ruby-mail is blocked at 2.6.4 to avoid breaking Banya (which is probably broken anyway). I need to evaluate 2.7.1 if we decide to keep it.

#10

Updated by Marc Dequènes 2 months ago

#11

Updated by Marc Dequènes about 2 months ago

The patch for redmine activities (thanks Pilou, see #655):

--- /dev/null   2015-12-14 19:43:03.976709288 +0100
+++ app/views/activities/index.api.rsb  2016-07-07 11:36:32.000000000 +0200
@@ -0,0 +1,13 @@
+api.array :activities do
+  @events.uniq.each do |event|
+    api.activity do
+      api.id          "#{event.class.name}/#{event.id}" 
+      api.title       event.event_title
+      api.type        event.event_type
+      api.project(:id => event.project.id, :name => event.project.name) unless event.project.nil?
+      api.description event.event_description
+      api.url         url_for(event.event_url)
+      api.created_on  event.event_datetime
+    end
+  end
+end
--- ./app/controllers/activities_controller.rb.orig     2016-10-10 09:48:13.000000000 +0200
+++ ./app/controllers/activities_controller.rb  2017-07-19 20:22:19.000000000 +0200
@@ -54,6 +54,7 @@
     end

     events = @activity.events(@date_from, @date_to)
+    @events = events # for API

     if events.empty? || stale?(:etag => [@activity.scope, @date_to, @date_from, @with_subprojects, @author, events.first, events.size, User.current, current_language])
       respond_to do |format|
@@ -61,6 +62,7 @@
           @events_by_day = events.group_by {|event| User.current.time_to_date(event.event_datetime)}
           render :layout => false if request.xhr?
         }
+        format.api
         format.atom {
           title = l(:label_activity)
           if @author

I found another patch repo_controller.patch but it's obsolete and the corresponding files were not found modified before switchign to Buster so this confirms it is no longer needed.

Also available in: Atom PDF