Project

General

Profile

Enhancement #501

Migration from owncloud to nextcloud

Added by Pierre-Louis Bonicoli over 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Category:
Service :: Groupware
Start date:
2016-11-25
Due date:
% Done:

100%

Patch Available:
Confirmed:
No
Branch:
Entity:
DuckCorp
Security:
Help Needed:

Description

We should migrate from owncloud to nextcloud.

Quotes from https://en.wikipedia.org/wiki/Nextcloud:

Nextcloud is an actively maintained fork of ownCloud.

While ownCloud offers an open-source community edition, they also offer a proprietary Enterprise Edition with additional features and support subscriptions. In comparison, Nextcloud is completely open source.

the amount of Nextcloud contributions quickly overtook ownClouds numbers as can be seen by GitHub statistics

Documentation:
https://nextcloud.com/migration/

stuffcloud_fixes.log View (1.44 KB) Marc Dequènes, 2017-04-20 16:31

History

#1 Updated by Pierre-Louis Bonicoli over 2 years ago

At FOSDEM, the Nextcloud team told me there isn't an official Debian package yet (and they are looking for help on this subject).

#2 Updated by Marc Dequènes over 2 years ago

  • Status changed from New to In Progress
  • Assignee set to Marc Dequènes
  • % Done changed from 0 to 90

So, migration happened: OC7 -> 8.0 -> 8.1 -> 8.2 -> NC9 -> 10 -> 11
A few config/DB manual fixes were necessary. We are using the provided 3rdparty libraries to be sure to have the expected dependencies.

Waiting for user reports.

config and storage needs to be moved to proper path in order to dissociate from the previous packaging. Then the packaging can be removed.

#3 Updated by Marc Dequènes over 2 years ago

config and storage moved/renamed.

old package with deps are removed too.

investigating an empty "shared with you" on 'boni' account (does not seem to affect any other user sharing).

Pilou is investigating sharing problems using a remote NC.

#4 Updated by Marc Dequènes over 2 years ago

Pilou found broken permissions on the /var/local/stuffcloud-data/*/files/Shared/ files; once fixed the problem with the 'boni' account was solved. Thanks Pilou.

#5 Updated by Marc Dequènes over 2 years ago

As for the remote NC, a new setting needed to be activated, so no problem.

#6 Updated by Pierre-Louis Bonicoli over 2 years ago

ACL before modifications:

# file: var/local/stuffcloud-data/alice/files/Shared
# owner: www-data
# group: www-data
user::rwx
group::rwx
other::---

# file: var/local/stuffcloud-data/arnau/files/Shared
# owner: www-data
# group: www-data
# flags: -s-
user::rwx
user:arnau:rwx
group::r-x
group:www-data:rwx
mask::rwx
other::r-x
default:user::rwx
default:user:arnau:rwx
default:group::r-x
default:group:www-data:rwx
default:mask::rwx
default:other::r-x

# file: var/local/stuffcloud-data/boni/files/Shared
# owner: www-data
# group: www-data
user::rwx
group::rwx
other::---
default:user::rwx
default:group::rwx
default:other::---

# file: var/local/stuffcloud-data/boutil/files/Shared
# owner: www-data
# group: www-data
user::rwx
group::rwx
other::---
default:user::rwx
default:group::rwx
default:other::---

# file: var/local/stuffcloud-data/duck/files/Shared
# owner: root
# group: root
user::rwx
group::r-x
other::r-x

# file: var/local/stuffcloud-data/duck-test_1226/files/Shared
# owner: www-data
# group: www-data
user::rwx
group::r-x
other::r-x

# file: var/local/stuffcloud-data/gorou/files/Shared
# owner: root
# group: www-data
# flags: -s-
user::rwx
group::rwx
other::---
default:user::rwx
default:group::rwx
default:other::---

# file: var/local/stuffcloud-data/jmb/files/Shared
# owner: www-data
# group: www-data
user::rwx
group::rwx
other::---
default:user::rwx
default:group::rwx
default:other::---

# file: var/local/stuffcloud-data/pilou/files/Shared
# owner: www-data
# group: www-data
user::rwx
group::rwx
other::---
default:user::rwx
default:group::rwx
default:other::---

After modifications, permissions were checked using find:
  • Owner and group:
    # find /var/local/stuffcloud-data/ ! -group www-data -o ! -user www-data
    /var/local/stuffcloud-data/lost+found
    
  • file mode bits (files)
    # find /var/local/stuffcloud-data/ \( \! -perm u=rw,g=r \) -a -type f
    /var/local/stuffcloud-data/appdata_occ74d2d09f0/avatar/duck/avatar.32.jpg
    /var/local/stuffcloud-data/appdata_occ74d2d09f0/avatar/duck/avatar.jpg
    
  • file mode bits (directories)
    # find /var/local/stuffcloud-data/ \( \! -perm u=rwx,g=rx \) -a -type d
    # no result
    

#7 Updated by Marc Dequènes over 2 years ago

Activated short URLs using some rewrite. README updated (until there is a real doc).

#8 Updated by Pierre-Louis Bonicoli over 2 years ago

Custom App Directories is now used:
1. a dedicated writable directory is used for users apps (/usr/local/share/stuffcloud-apps/) (/srv/www/sites/milkypond/stuff.milkypond.org/stuffcloud/apps2 is a symlink);
2. /srv/www/sites/milkypond/stuff.milkypond.org/stuffcloud/apps/ isn't writable anymore.

New configuration:

  array (
    0 =>
    array (
      'path' => '/srv/www/sites/milkypond/stuff.milkypond.org/stuffcloud/apps/',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/srv/www/sites/milkypond/stuff.milkypond.org/stuffcloud/apps2/',
      'url' => '/apps2',
      'writable' => true,
    ),
  ),

#9 Updated by Pierre-Louis Bonicoli over 2 years ago

Backtrace:

 u"0 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/SharedStorage.php(116): OCFilesFilesystem::initMountPoints('29827636-6ca4-1...')n",
 u'1 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/SharedStorage.php(449): OCAFiles_SharingSharedStorage->init()n',
 u'2 [internal function]: OCAFiles_SharingSharedStorage->getSourceStorage()n',
 u'3 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Storage/Wrapper/Wrapper.php(501): call_user_func_array(Array, Array)n',
 u"4 [internal function]: OCFilesStorageWrapperWrapper->__call('getSourceStorag...', Array)n",
 u'5 [internal function]: OCAFiles_TrashbinStorage->getSourceStorage()n',
 u'6 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Storage/Wrapper/Wrapper.php(501): call_user_func_array(Array, Array)n',
 u"7 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/Cache.php(69): OCFilesStorageWrapperWrapper->__call('getSourceStorag...', Array)n",
 u'8 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/Cache.php(69): OCAFiles_AntivirusAvirWrapper->getSourceStorage()n',
 u'9 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Cache/Wrapper/CacheJail.php(296): OCAFiles_SharingCache->getCache()n',
 u'10 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/View.php(1687): OCFilesCacheWrapperCacheJail->getPathById(47072)n',
 u'11 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/SharedStorage.php(117): OCFilesView->getPath(47072)n',
 u'12 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/SharedStorage.php(454): OCAFiles_SharingSharedStorage->init()n',
 u'13 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Storage/Wrapper/Wrapper.php(541): OCAFiles_SharingSharedStorage->getWrapperStorage()n',
 u"14 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Storage/Wrapper/Wrapper.php(541): OCFilesStorageWrapperWrapper->verifyPath('', 'Quotidien')n",
 u"15 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Storage/Wrapper/Wrapper.php(541): OCFilesStorageWrapperWrapper->verifyPath('', 'Quotidien')n",
 u"16 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/View.php(1796): OCFilesStorageWrapperWrapper->verifyPath('', 'Quotidien')n",
 u"17 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/dav/lib/Connector/Sabre/ObjectTree.php(127): OCFilesView->verifyPath('Shared/Quotidie...', 'Quotidien')n",
 u"18 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/3rdparty/sabre/dav/lib/DAV/Server.php(939): OCADAVConnectorSabreObjectTree->getNodeForPath('Shared/Quotidie...')n",
 u"19 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(336): SabreDAVServer->getPropertiesForPath('Shared/Quotidie...', Array, 1)n",
 u'20 [internal function]: SabreDAVCorePlugin->httpPropFind(Object(SabreHTTPRequest), Object(SabreHTTPResponse))n',
 u'21 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)n',
 u"22 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/3rdparty/sabre/dav/lib/DAV/Server.php(479): SabreEventEventEmitter->emit('method:PROPFIND', Array)n",
 u'23 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/3rdparty/sabre/dav/lib/DAV/Server.php(254): SabreDAVServer->invokeMethod(Object(SabreHTTPRequest), Object(SabreHTTPResponse))n',
 u'24 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/dav/appinfo/v1/webdav.php(60): SabreDAVServer->exec()n',
 u"25 /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/remote.php(165): require_once('/srv/www/sites/...')n",

Log messages:

{
  "reqId":"WLmbDsHIKrEAAFJ233UAAAAU",
  "remoteAddr":"79.88.186.54",
  "app":"files",
  "message":" Backends provided no user object for 29827636-6ca4-1032-90bf-157529423ee4",
  "level":3,
  "time":"2017-03-03T16:34:25+00:00",
  "method":"PROPFIND",
  "url":"\/remote.php\/webdav\/Shared\/Quotidien",
  "user":"boni",
  "version":"11.0.2.7" 
}
{
  "reqId":"WLmbDsHIKrEAAFJ233UAAAAU",
  "remoteAddr":"79.88.186.54",
  "app":"no app in context",
  "message":"Exception: {see above}",
  "level":3,"time":"2017-03-03T16:34:25+00:00",
  "method":"PROPFIND",
  "url":"\/remote.php\/webdav\/Shared\/Quotidien",
  "user":"boni",
  "version":"11.0.2.7" 
}

Related error caused by the previous exception:

[Sat Mar 04 00:51:10.572536 2017] [:error] [pid 26805] [client 90.61.236.215:50796] PHP Fatal error:  Call to a member function getCache() on null in /srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/Cache.php on line 69

The stack just before this related error PHP Fatal error:

[{u'args': [],
  u'class': u'OCAFiles_SharingCache',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/Cache/Wrapper/CacheJail.php',
  u'function': u'getCache',
  u'line': 296,
  u'object': {},
  u'type': u'->'},
 {u'args': [53142],
  u'class': u'OCFilesCacheWrapperCacheJail',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Files/View.php',
  u'function': u'getPathById',
  u'line': 1687,
  u'object': {},
  u'type': u'->'},
 {u'args': [53142],
  u'class': u'OCFilesView',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/activity/lib/FilesHooks.php',
  u'function': u'getPath',
  u'line': 784,
  u'object': {},
  u'type': u'->'},
 {u'args': [u'unshared_user_self', u'boni', 53142, u'folder'],
  u'class': u'OCAActivityFilesHooks',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/activity/lib/FilesHooks.php',
  u'function': u'shareNotificationForSharer',
  u'line': 605,
  u'object': {},
  u'type': u'->'},
 {u'args': [u'boni', 53142, u'folder', u'/Shared/Quotidien', False],
  u'class': u'OCAActivityFilesHooks',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/activity/lib/FilesHooks.php',
  u'function': u'shareFileOrFolderWithUser',
  u'line': 575,
  u'object': {},
  u'type': u'->'},
 {u'args': [{u'fileSource': 53142,
    u'fileTarget': u'/Shared/Quotidien',
    u'id': u'583',
    u'itemSource': 53142,
    u'itemType': u'folder',
    u'itemparent': None,
    u'shareType': 0,
    u'shareWith': u'boni',
    u'uidOwner': u'pilou'}],
  u'class': u'OCAActivityFilesHooks',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/activity/lib/FilesHooksStatic.php',
  u'function': u'unShare',
  u'line': 101,
  u'object': {},
  u'type': u'->'},
 {u'args': [{u'fileSource': 53142,
    u'fileTarget': u'/Shared/Quotidien',
    u'id': u'583',
    u'itemSource': 53142,
    u'itemType': u'folder',
    u'itemparent': None,
    u'shareType': 0,
    u'shareWith': u'boni',
    u'uidOwner': u'pilou'}],
  u'class': u'OCAActivityFilesHooksStatic',
  u'function': u'unShare',
  u'type': u'::'},
 {u'args': [[u'OCAActivityFilesHooksStatic', u'unShare'],
   {u'fileSource': 53142,
    u'fileTarget': u'/Shared/Quotidien',
    u'id': u'583',
    u'itemSource': 53142,
    u'itemType': u'folder',
    u'itemparent': None,
    u'shareType': 0,
    u'shareWith': u'boni',
    u'uidOwner': u'pilou'}],
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/legacy/hook.php',
  u'function': u'call_user_func',
  u'line': 106},
 {u'args': [u'OCPShare',
   u'pre_unshare',
   {u'fileSource': 53142,
    u'fileTarget': u'/Shared/Quotidien',
    u'id': u'583',
    u'itemSource': 53142,
    u'itemType': u'folder',
    u'itemparent': None,
    u'shareType': 0,
    u'shareWith': u'boni',
    u'uidOwner': u'pilou'}],
  u'class': u'OC_Hook',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Share20/Manager.php',
  u'function': u'emit',
  u'line': 836,
  u'type': u'::'},
 {u'args': [{}],
  u'class': u'OCShare20Manager',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/apps/files_sharing/lib/Controller/ShareAPIController.php',
  u'function': u'deleteShare',
  u'line': 282,
  u'object': {},
  u'type': u'->'},
 {u'args': [u'583'],
  u'class': u'OCAFiles_SharingControllerShareAPIController',
  u'function': u'deleteShare',
  u'object': {},
  u'type': u'->'},
 {u'args': [[{}, u'deleteShare'], [u'583']],
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/AppFramework/Http/Dispatcher.php',
  u'function': u'call_user_func_array',
  u'line': 160},
 {u'args': [{}, u'deleteShare'],
  u'class': u'OCAppFrameworkHttpDispatcher',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/AppFramework/Http/Dispatcher.php',
  u'function': u'executeController',
  u'line': 90,
  u'object': {},
  u'type': u'->'},
 {u'args': [{}, u'deleteShare'],
  u'class': u'OCAppFrameworkHttpDispatcher',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/AppFramework/App.php',
  u'function': u'dispatch',
  u'line': 114,
  u'object': {},
  u'type': u'->'},
 {u'args': [u'ShareAPIController',
   u'deleteShare',
   {},
   {u'_route': u'ocs.files_sharing.ShareAPI.deleteShare', u'id': u'583'}],
  u'class': u'OCAppFrameworkApp',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/AppFramework/Routing/RouteActionHandler.php',
  u'function': u'main',
  u'line': 47,
  u'type': u'::'},
 {u'args': [{u'_route': u'ocs.files_sharing.ShareAPI.deleteShare',
    u'id': u'583'}],
  u'class': u'OCAppFrameworkRoutingRouteActionHandler',
  u'function': u'__invoke',
  u'object': {},
  u'type': u'->'},
 {u'args': [{},
   {u'_route': u'ocs.files_sharing.ShareAPI.deleteShare', u'id': u'583'}],
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/lib/private/Route/Router.php',
  u'function': u'call_user_func',
  u'line': 299},
 {u'args': [u'/ocsapp/apps/files_sharing/api/v1/shares/583'],
  u'class': u'OCRouteRouter',
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/ocs/v1.php',
  u'function': u'match',
  u'line': 79,
  u'object': {},
  u'type': u'->'},
 {u'args': [u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/ocs/v1.php'],
  u'file': u'/srv/www/sites/milkypond/stuff.milkypond.org/nextcloud-11.0.2/ocs/v2.php',
  u'function': u'require_once',
  u'line': 23}

#10 Updated by Marc Dequènes over 2 years ago

Made fixes around storage and filecache. Fixed Shared folder and related cache crashes.

#11 Updated by Marc Dequènes about 2 years ago

  • % Done changed from 90 to 80

I upgraded to NC12. Not sure the remaining problems are solved. The speed issue clearly is not (or maybe worse).

Also it added new problems:
- files view is broken
- X-Content-Type-Options "nosniff" and some bad MIME type on two apps/files URL: disabling the security option but I'm not sure what is better afterwards, so reverted
- https://github.com/nextcloud/server/issues/4857: probably responsible for the view mess, but was not able to solve it yet

#12 Updated by Marc Dequènes about 2 years ago

I forgot to say the piwik plugin is not yet compatible :-/.

I discovered in oc_storages there is still reference to the old data path (ID: 11). So I wanted to converted oc_mounts from it to the new one (ID: 46) but there was none. I then removed the old storage.

I'm digging around ResourceLocator and SCSSCacher but not found the problem yet. The CSS are properly generated from the SCSS in `/var/local/stuffcloud-data/appdata_occ74d2d09f0/` but it is not found by the ResourceLocator.

#13 Updated by Marc Dequènes about 2 years ago

This ticket helped: https://github.com/nextcloud/server/issues/5057

So the ResourceLocator is confused by the curren version symlink (stuffcloud->nextcloud-12.0.0) so I temporarily inverted the symlink and it works fine again.

So it should be fixed in 12.0.1 I guess.

#14 Updated by Marc Dequènes about 2 years ago

As for the piwik plugin, the author replied to the ticket he would fix this asap: https://github.com/sualko/owncloud_piwik/issues/32

#15 Updated by Marc Dequènes about 2 years ago

  • % Done changed from 80 to 90

So, linking with NC10 on indie.host worked said Pilou.

#16 Updated by Marc Dequènes about 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

So the piwik app was updated.

So this migration is complete now.

Also available in: Atom PDF