This site has been archived and you can no longer log in or post new messages. For up-to-date community resources please visit ezplatform.com

eZ Community » Forums » Developer » Process to work with 20 developers
expandshrink

Process to work with 20 developers

Process to work with 20 developers

Thursday 01 December 2011 10:14:47 am - 11 replies

Hi,

I think I have a big topic here. We are working with 20 developers on the same project, each developer is using his own VM to work on his local database. The goal is, when committing and updating with SVN, automatically export / import on each local database with new classes and also roles. I don't know if this process has been done before, but I think this question is a hard topic.

Thanks,

Jean-Luc.

Thursday 01 December 2011 1:01:22 pm

Yes, that is a very important issue: staging in general. Files are easy (git, svn, etc.) to stage but the database is problematic because ez stores some info there (like classes, etc).

Standard development situation is that you have a central development installation and a staging installation (after going live there is also the production installation). We sync files via some code repository system but that leaves us with manual database changes. Usually one installation is declared as a "master" meaning that all database changes should be first done there to avoid conflict. Changes are migrated in 2 ways: for new classes by packages, for existing classes manually. Roles are manual too.

I am thinking about a staging extension for a long time (never had time to develop it) which would at least pull stuff (classes, roles, sections, etc.) from a "master" instance into another one.

Modified on Thursday 01 December 2011 1:02:07 pm by Ivo Lukač

Thursday 01 December 2011 1:25:46 pm

Hi Ivo,

Thanks for your answer. Actually, this is the way I usually work. The main goal is to automate the "manual" tasks, such as creating / updating a class (yes, there is the package module) or creating / updating a role. I guess this is not doable?

Thanks,

Jean-Luc.

Thursday 01 December 2011 1:44:55 pm

Hi Ivo,

Thanks for your answer. Actually, this is the way I usually work. The main goal is to automate the "manual" tasks, such as creating / updating a class (yes, there is the package module) or creating / updating a role. I guess this is not doable?

Thanks,

Jean-Luc.

I wouldn't say it's not doable happy.gif Emoticon 

It should be even straightforward to make an extension which can replicate some rows for some tables on database level.

Things get complicated when you want this replication to go 2-way. Then you need to solve conflicts and that can be really nasty to develop happy.gif Emoticon

Thursday 01 December 2011 2:04:43 pm

Hi

Maybe this extension can help:

http://projects.ez.no/ezxmlinstaller

Thursday 01 December 2011 2:09:05 pm

Hi

Maybe this extension can help:

http://projects.ez.no/ezxmlinstaller

Interesting, but it just has ezcreateclass and no ezupdateclass sad.gif Emoticon

Thursday 01 December 2011 2:25:23 pm

Hi

Maybe this extension can help:

http://projects.ez.no/ezxmlinstaller

Interesting, but it just has ezcreateclass and no ezupdateclass sad.gif Emoticon

Hi,

I'll check this extension and maybe will extend it. Thanks for your help!

Thursday 01 December 2011 2:38:06 pm

 

I'll check this extension and maybe will extend it. Thanks for your help!

Don't forget to contribute back blunk.gif Emoticon

Thursday 01 December 2011 4:47:17 pm

What we did at the last place was have a svn repository on a dev machine and the everyone had to have their own version of the site on their own laptop.  Then when database changes were made - if it was to be persistent they would do it on the dev  machine but if it was just to test something then they would do it local and it would get wiped out the next time they got the master database from the dev machine.

This was with 5 people and it mostly worked but sometimes it wouldn't - such as if someone started editing on the dev machine as themselves (or worse, root) it would normally screw up the permissions so then commits would stop working for everyone.  It probably would work better with github, dunno, I was thinking of migrating before I went out on my own.

We also kept the settings directory out of svn so that would slowly get out of sync which was always a mess.

Anyway - here is the shell script that the developer would have on their machine to sync from the master - get the database, load it locally and sync the var files.  It doesn't work with multi-database sites (it could with modifications and I did a version where it would use a siteaccess, if you're using windows with cygwin there are also modifications necessary - although I don't remember what they are)  - you can run this as root but you don't have to but then you'll get a chmod error on the rsync.  If you do put passwords in the file then you'll want to change the permissions as appropriate too:

 ## Copyright (C) 2011 Leiden Tech <nospam@leidentech.com>#### This program is free software; you can redistribute it and/or modify## it under the terms of the GNU General Public License as published by## the Free Software Foundation; either version 3 of the License, or## (at your option) any later version.#### This program is distributed in the hope that it will be useful,## but WITHOUT ANY WARRANTY; without even the implied warranty of## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the## GNU General Public License for more details.#### You should have received a copy of the GNU General Public License## along with this program; if not, write to the Free Software## Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA##
#!/bin/bash
remote_machine=
port=
#if these aren't set they will be set using ini values
remote_db_user=
remote_db_passwd=
remote_db=
local_db_user=
local_db_passwd=
local_db=
remote_storage_path=
local_storage_path=
 
#end user set variables
 
cmd=`basename $0`
dofiles="true"
dodb="true"
skipdl="false"
checkonly="false"
 
#so we don't need passwords in this file
getinidbinfo ()
{
dbinfo=`php -r "require 'autoload.php';
\\$script = eZScript::instance( array( 'debug-message' => '', 'use-session' => true, 'use-modules' => true, 'use-extensions' => true ) );
\\$script->startup();
\\$script->setUseSiteAccess( '${1}' );
\\$script->initialize();
\\$ini = eZINI::instance();
\\$database=\\$ini->variable( 'DatabaseSettings', 'Database' );
\\$user=\\$ini->variable( 'DatabaseSettings', 'User' );
\\$password=\\$ini->variable( 'DatabaseSettings', 'Password' );
\\$script->shutdown();
echo \\$database.':'.\\$user.':'.\\$password;"`
#echo $dbinfo DBINFO
remote_db=${remote_db:-`echo $dbinfo|cut -f1 -d":"`}
remote_db_user=${remote_db_user:-`echo $dbinfo|cut -f2 -d":"`}
remote_db_passwd=${remote_db_passwd:-`echo $dbinfo|cut -f3 -d":"`}
}
getinidbinfo $1
getvardir ()
{
vardir=`php -r "require 'autoload.php';
\\$script = eZScript::instance( array( 'debug-message' => '', 'use-session' => true, 'use-modules' => true, 'use-extensions' => true ) );
\\$script->startup();
\\$script->setUseSiteAccess( '${1}' );
\\$script->initialize();
\\$ini = eZINI::instance();
\\$vardir=\\$ini->variable( 'FileSettings', 'VarDir' );
\\$script->shutdown();
echo \\$vardir;"`
echo $vardir
}
 
checkarg ()
{
funclist=`grep '()' $cmd|egrep -v "checkarg"|sed 's/()//'`
for func in $*
do
        if [ -z "`echo $funclist|grep ${func}`" ]
        then
                echo "invalid arg: $func"
                echo -e "valid args:\n$funclist"
        else
                $func
        fi
done
}
 
prthelp ()
{
cat <<-EOF
USAGE $cmd [-d] [-f]
        -d only database
        -f only files
EOF
exit
}
 
##############
#                        #
#  MAIN              #
#                        #
##############
 
#sane defaults
remote_storage_path=$PWD/`getvardir`/storage/
local_storage_path=`getvardir`/storage
remote_user=${remote_user:-`whoami`}
local_user=${local_user:-$remote_user}
local_db_user=${local_db_user:-$remote_db_user}
local_db=${local_db:-$remote_db}
local_db_passwd=${local_db_passwd:-$remote_db_passwd}
 
for opt
do
     case $opt in
          "-c"|"--check")    checkonly="true";;
          "-d"|"--database")    dofiles="false";;
          "-f"|"--files")       dodb="false";;
          "-h"|"--help")        prthelp;;
          *)    echo checkarg ${*:-\"\"};;
     esac
done
if [ $checkonly = "true" ]
then
        echo "remote_machine $remote_machine"
        echo "local_db $local_db"
        echo "local_db_user $local_db_user"
        echo "local_db_passwd $local_db_passwd"
        echo "local_storage_path $local_storage_path"
        exit
fi
if [ $dodb = "true" ]
then
        if [ -f "/home/${local_user}|>/${local_db}|>.sql.gz" ]
        then
                echo -e "${local_db}|>.sql.gz exists, redownload? \c"
                read answer
                if [ `echo $answer|egrep -ic "^Y$|^YES$"` -eq 0 ]
                then #NOT YES
                        skipdl="true"
                fi
                #if file exists, should ask if should download
        fi
        if [ "$skipdl" = "false" ]
        then
                echo "Redownloading"
                ssh ${port:+-p $port} ${remote_user}@${remote_machine} -C "mysqldump --opt --skip-lock-tables -u ${remote_db_user}|> --password=${remote_db_passwd}|> ${remote_db}|>|gzip > /home/${remote_user}|>/${remote_db}|>.sql.gz"
                scp ${port:+-P $port} ${remote_user}@${remote_machine}:/home/${remote_user}/${remote_db}.sql.gz /home/${local_user}/${local_db}.sql.gz
        else
                echo "Skipping download"
        fi
        #should be recreate database - dunno, maybe should be an option
        mysql -u ${local_db_user} --password=${local_db_passwd} -e "DROP DATABASE ${local_db}|>;CREATE DATABASE ${local_db}|> DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;"
        gunzip < /home/${local_user}/${local_db}.sql.gz|mysql -u ${local_db_user} --password=${local_db_passwd} ${local_db}
fi
if [ "$dofiles" = "true" ]
then
        rsync -avH ${port:+-e "ssh -p $port"} ${remote_user}@${remote_machine}:${remote_storage_path} ${local_storage_path}
fi

Thursday 01 December 2011 5:48:37 pm

Hi Ivo,

Thanks for your answer. Actually, this is the way I usually work. The main goal is to automate the "manual" tasks, such as creating / updating a class (yes, there is the package module) or creating / updating a role. I guess this is not doable?

Thanks,

Jean-Luc.

I wouldn't say it's not doable happy.gif Emoticon 

It should be even straightforward to make an extension which can replicate some rows for some tables on database level.

Things get complicated when you want this replication to go 2-way. Then you need to solve conflicts and that can be really nasty to develop happy.gif Emoticon

things get complicated even more when you intend to do that on a dev database and a production one, specially if your customer have some knowledge about adding new roles or things like that... 

Modified on Thursday 01 December 2011 5:49:07 pm by Carlos Revillo

Thursday 01 December 2011 6:08:06 pm

Hi

Maybe this extension can help:

http://projects.ez.no/ezxmlinstaller

Interesting, but it just has ezcreateclass and no ezupdateclass sad.gif Emoticon

@Ivo:

Actually, browsing quickly the ezcreateclass handler, you can replace or extend the class if it is currently existing.

@Nicolas:

Of course!

Thursday 01 December 2011 7:34:05 pm

If anyone is interested in providing code that diffs content class definition and generates the neecssary update operations, please do it either in the ezxmlinstaller or as pull request for the packaging system. We really need this feature to be in the core of the product!

expandshrink

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu

Proudly Developed with from