Difference between revisions of "Development environment"

From Archivematica
Jump to navigation Jump to search
m (Remove dev docs category (still in subcategory deprecated))
 
(36 intermediate revisions by 9 users not shown)
Line 1: Line 1:
 
[[Main Page]] > [[Development]] > Development Environment
 
[[Main Page]] > [[Development]] > Development Environment
  
This page explains how you can configure and use a Ubuntu Linux system, to use as an Archivematica development environment.
+
{| class="wikitable" style="background-color:#ffcccc; font-size: 120%; font-weight: bold; " cellpadding="10"
 +
| Please note: Development environment instructions were for Archivematica 1.4 and earlier - these notes are no longer supported. Instead please see: [[Getting started]]
 +
|}
 +
 
 +
This page explains how you can configure and use a standard Linux system as an Archivematica development environment.
 
The Archivematica development environment is available for developers that want the ability to customize or enhance their own Archivematica installation and/or [[contribute code]] back to the Archivematica project.
 
The Archivematica development environment is available for developers that want the ability to customize or enhance their own Archivematica installation and/or [[contribute code]] back to the Archivematica project.
  
 
=Setup=
 
=Setup=
*Use [[Subversion]] to checkout Archivematica code
+
*Install ubuntu 12.04
**If you have commit access, use the private repository
+
*create a non-root user (with sudo privileges)
**If you have read-only access, use the public [http://code.google.com/p/archivematica/source/browse/trunk#trunk/includes/archivematica GoogleCode repository]
+
*log in as your new non-root user
<pre>svn checkout http://archivematica.googlecode.com/svn/trunk/ archivematica</pre>
+
*install archivematica storage service
 +
<pre>
 +
sudo apt-get update
 +
sudo apt-get install python-software-properties
 +
sudo add-apt-repository ppa:archivematica/release
 +
sudo add-apt-repository ppa:archivematica/externals
 +
sudo apt-get update
 +
sudo apt-get upgrade
 +
sudo apt-get install archivematica-storage-service
 +
sudo rm /etc/nginx/sites-enabled/default
 +
sudo ln -s /etc/nginx/sites-available/storage /etc/nginx/sites-enabled/storage
 +
sudo ln -s /etc/uwsgi/apps-available/storage.ini /etc/uwsgi/apps-enabled/storage.ini
 +
sudo service uwsgi restart
 +
sudo service nginx restart
 +
</pre>
  
 +
*install git
 +
<pre>sudo apt-get install git</pre>
  
*Run localDevSetup scripts.
+
*Use git to checkout Archivematica code
** <pre>cd archivematica/localDevSetup/</pre>
+
<pre>git clone https://github.com/artefactual/archivematica.git</pre>
** <pre>./installLocalDevRequirements.sh</pre>
 
** <pre>./createArchivematicaUserAndGroupForLocalDev.sh</pre>
 
** <pre>./createLocalDevDirectories.sh</pre>
 
** <pre>cd createDatabases</pre>
 
** <pre>./postBuildRun.sh</pre>
 
**This will create links in your local Ubuntu environment back to the relevant Archivematica scripts in your SVN checkout, allowing you to run a local Archivematica version which can continue to pull down revision updates from the code repository
 
**If you have commit access, you can work on the files in the checkout directory, make changes and commit those back to SVN repository, while also running Archivematica locally to test your changes and commits from other developers.
 
  
*Additional local environment configuration
+
*Run the install and helper scripts.
** Add Crontab for quarantine check
+
** <pre>cd archivematica</pre>
** <pre>sudo crontab -e</pre>
+
** <pre>./dev-installer</pre>
** <pre>* * * * * flock -xn /var/lock/quarantine.lock /usr/lib/archivematica/MCPServer/delay/delay.py 60 "/var/archivematica/sharedDirectory/watchedDirectories/quarantined"</pre>
+
** Answer Y to all prompts.
 +
** Restart the machine. (this is to enable the upstart services).
 +
** <pre>cd archivematica</pre>
 +
** <pre>./dev-helper</pre>
 +
** Answer Y to all prompts.
 +
** Complete AtoM setup
 +
*** The database should already be created.
 +
*** http://localhost/atom
 +
*** https://www.qubit-toolkit.org/wiki/Installation#Open_Qubit.2C_ICA-AtoM.2C_or_DCB_in_your_web_browser
 +
** Open the [dashboard http://localhost]
  
=Run=
+
=Configuring and Using Archivematica=
*Start Archivematica MCP Server & Client
+
These manuals contain information about the use and configuration of Archivematica:
**Open a terminal and type in the following commands:
+
<br/>https://www.archivematica.org/wiki/Administrator_manual
***<pre>sudo service openofficed stop</pre>
+
<br/>https://www.archivematica.org/wiki/User_Manual
***<pre>sudo service openofficed start</pre>
 
***<pre>sudo service archivematica-mcp-serverd restart</pre>
 
***<pre>sudo service archivematica-mcp-clientd restart</pre>
 
  
*To start processing a SIP:
+
=Update=
**Open Thunar file manager
 
**if using sample SIPs, 'export' the SIP directory
 
***<pre>svn export buildVM/includes/sampledata ~/sampledata</pre>
 
**'cut' the SIP directory and paste into the receiveSIP directory
 
**Open the Firefox browser and type 'localhost' as the address to view SIP processing progress via the Dashboard
 
  
=Update=
+
To pull down the latest code commits from the repository, and reset your dev install, navigate to the directory where Archivematica has been cloned:
To pull down the latest code commits from the repository, navigate to the directory where Archivematica trunk has been checked out:
+
*Change Directory to the archivematica git directory.
*Change Directory to the archivematica SVN directory.
 
 
<pre>cd ~/archivematica/</pre>
 
<pre>cd ~/archivematica/</pre>
*<pre>cd archivematica/localDevSetup/</pre>
+
*Check to see if you have any local changes that need to be stashed
*<pre>./cleanup.sh</pre>
+
<pre>git diff</pre>
*<pre>cd ~/archivematica/</pre>
+
*If there are local changes please stash or commit them. (Or you won't be able to update).
*<pre>svn up</pre>
+
<pre>git stash</pre>
*<pre>cd ~/archivematica/localDevSetup/</pre>
+
Update...
*<pre>./createLocalDevDirectories.sh</pre>
+
*<pre>./dev-helper</pre>
*Note: changes to the MCP database schema may have occurred. To make these changes to your local database run the following command.
+
*<pre>"Would you like to git pull?" (y/N) y</pre>
'''THIS WILL ERASE THE DATA ON SIPs CURRENTLY IN YOUR DATABASE.'''
+
*<pre>"Would you like to update/install package requirements?" (y/N) </pre>
*<pre>./recreateDB.sh</pre>
+
*<pre>"Would you like to recreate the databases?" (y/N) y</pre>
 +
**Note: this command will remove any processing sips/transfers.
 +
*<pre>"Would you like to erase the ElasticSearch indexes?" (y/N) </pre>
 +
*<pre>"Would you like to clear transfer backlog and AIP storage?" (y/N) </pre>
 +
**Note: this command will delete all stored AIPs and stored backlogged transfers.
 +
*<pre>"Would you like to restart archivematica services?" (y/N) y</pre>
 +
*<pre>"Would you like to update sample data in /home/user/archivematica-sampledata?" (y/N) </pre>
 +
*<pre>"Would you like to export sample data from /home/user/archivematica-sampledata?" (y/N) </pre>
 +
*<pre>"Would you like to update AtoM and restart its atom-worker service?" (y/N) y</pre>
 +
If you stashed changes, re-apply them with
 +
<pre>git stash pop</pre>
  
 
=Troubleshooting=
 
=Troubleshooting=
 +
* If it stalls during update "Would you like to update/install package requirements?" (y/N)
 +
** Stop the script with CTRL + C
 +
** Try installing the item it failed on on the command line. Ie.
 +
** <pre>sudo apt-get install postfix</pre>
 +
** Restart the dev-helper "Would you like to update/install package requirements?"
 +
 +
* If you get this error error starting service:
 +
** <pre>sudo start archivematica-mcp-server</pre>
 +
** <pre>start: Unknown job: archivematica-mcp-server</pre>
 +
** Reload the configuration  [http://askubuntu.com/a/291644 Original answer] and try again
 +
** <pre>sudo initctl reload-configuration</pre>
 +
** Alternatively, reboot the machine.
 +
 
*If a SIP processing fails, it will move it to the 'failed' directory which is located:
 
*If a SIP processing fails, it will move it to the 'failed' directory which is located:
**<pre>var/archivematica/sharedDirectory/watchedDirectories/failed</pre>
+
**<pre>/var/archivematica/sharedDirectory/watchedDirectories/failed</pre>
  
 
*if ArchivematicaServer freezes  
 
*if ArchivematicaServer freezes  
**<pre>sudo service archivematica-mcp-serverd restart</pre>
+
**<pre>sudo restart archivematica-mcp-server</pre>
**<pre>sudo service archivematica-mcp-clientd restart</pre>
+
**<pre>sudo restart archivematica-mcp-client</pre>
  
 
*if ArchivematicaClient freezes (in terminal kill command)
 
*if ArchivematicaClient freezes (in terminal kill command)
**<pre>sudo service archivematica-mcp-clientd restart</pre>
+
**<pre>sudo restart archivematica-mcp-client</pre>
  
 
*Updates to the Dashboard may require an Apache webserver restart:
 
*Updates to the Dashboard may require an Apache webserver restart:
Line 70: Line 106:
  
 
*If you find a problem running the Dashboard and you want to get a detailed error log to report us, please switch it to debug mode following [http://archivematica.org/wiki/index.php?title=Dashboard#Debug_mode these instructions].
 
*If you find a problem running the Dashboard and you want to get a detailed error log to report us, please switch it to debug mode following [http://archivematica.org/wiki/index.php?title=Dashboard#Debug_mode these instructions].
 +
 +
*MCP is currently logging to the /tmp/directory
 +
** /tmp/archivematicaMCPClient-HOST.log
 +
** /tmp/archivematicaMCPServer-HOST-DATE.log
 +
** /tmp/archivematicaMCPServerPID
 +
 +
[[Category:Deprecated]]

Latest revision as of 13:23, 23 March 2017

Main Page > Development > Development Environment

Please note: Development environment instructions were for Archivematica 1.4 and earlier - these notes are no longer supported. Instead please see: Getting started

This page explains how you can configure and use a standard Linux system as an Archivematica development environment. The Archivematica development environment is available for developers that want the ability to customize or enhance their own Archivematica installation and/or contribute code back to the Archivematica project.

Setup[edit]

  • Install ubuntu 12.04
  • create a non-root user (with sudo privileges)
  • log in as your new non-root user
  • install archivematica storage service
sudo apt-get update
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:archivematica/release
sudo add-apt-repository ppa:archivematica/externals
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install archivematica-storage-service
sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/storage /etc/nginx/sites-enabled/storage
sudo ln -s /etc/uwsgi/apps-available/storage.ini /etc/uwsgi/apps-enabled/storage.ini
sudo service uwsgi restart
sudo service nginx restart
  • install git
sudo apt-get install git
  • Use git to checkout Archivematica code
git clone https://github.com/artefactual/archivematica.git

Configuring and Using Archivematica[edit]

These manuals contain information about the use and configuration of Archivematica:
https://www.archivematica.org/wiki/Administrator_manual
https://www.archivematica.org/wiki/User_Manual

Update[edit]

To pull down the latest code commits from the repository, and reset your dev install, navigate to the directory where Archivematica has been cloned:

  • Change Directory to the archivematica git directory.
cd ~/archivematica/
  • Check to see if you have any local changes that need to be stashed
git diff
  • If there are local changes please stash or commit them. (Or you won't be able to update).
git stash

Update...

  • ./dev-helper
  • "Would you like to git pull?" (y/N) y
  • "Would you like to update/install package requirements?" (y/N) 
  • "Would you like to recreate the databases?" (y/N) y
    • Note: this command will remove any processing sips/transfers.
  • "Would you like to erase the ElasticSearch indexes?" (y/N) 
  • "Would you like to clear transfer backlog and AIP storage?" (y/N) 
    • Note: this command will delete all stored AIPs and stored backlogged transfers.
  • "Would you like to restart archivematica services?" (y/N) y
  • "Would you like to update sample data in /home/user/archivematica-sampledata?" (y/N) 
  • "Would you like to export sample data from /home/user/archivematica-sampledata?" (y/N) 
  • "Would you like to update AtoM and restart its atom-worker service?" (y/N) y

If you stashed changes, re-apply them with

git stash pop

Troubleshooting[edit]

  • If it stalls during update "Would you like to update/install package requirements?" (y/N)
    • Stop the script with CTRL + C
    • Try installing the item it failed on on the command line. Ie.
    • sudo apt-get install postfix
    • Restart the dev-helper "Would you like to update/install package requirements?"
  • If you get this error error starting service:
    • sudo start archivematica-mcp-server
    • start: Unknown job: archivematica-mcp-server
    • Reload the configuration Original answer and try again
    • sudo initctl reload-configuration
    • Alternatively, reboot the machine.
  • If a SIP processing fails, it will move it to the 'failed' directory which is located:
    • /var/archivematica/sharedDirectory/watchedDirectories/failed
  • if ArchivematicaServer freezes
    • sudo restart archivematica-mcp-server
    • sudo restart archivematica-mcp-client
  • if ArchivematicaClient freezes (in terminal kill command)
    • sudo restart archivematica-mcp-client
  • Updates to the Dashboard may require an Apache webserver restart:
    • sudo /etc/init.d/apache2 restart
  • If you find a problem running the Dashboard and you want to get a detailed error log to report us, please switch it to debug mode following these instructions.
  • MCP is currently logging to the /tmp/directory
    • /tmp/archivematicaMCPClient-HOST.log
    • /tmp/archivematicaMCPServer-HOST-DATE.log
    • /tmp/archivematicaMCPServerPID