Development environment

From Archivematica
(Difference between revisions)
Jump to: navigation, search
(Setup: Editing cron to reflect changes.)
m (Remove dev docs category (still in subcategory deprecated))
 
(53 intermediate revisions by 9 users not shown)
Line 1: Line 1:
 
[[Main Page]] > [[Development]] > Development Environment
 
[[Main Page]] > [[Development]] > Development Environment
  
This page explains how you can configure and use an Ubuntu Linux system to use as a basic Archivematica development environment.
+
{| class="wikitable" style="background-color:#ffcccc; font-size: 120%; font-weight: bold; " cellpadding="10"
 +
| Please note: Development environment instructions were for Archivematica 1.4 and earlier - these notes are no longer supported. Instead please see: [[Getting started]]
 +
|}
 +
 
 +
This page explains how you can configure and use a standard Linux system as an Archivematica development environment.
 +
The Archivematica development environment is available for developers that want the ability to customize or enhance their own Archivematica installation and/or [[contribute code]] back to the Archivematica project.
  
 
=Setup=
 
=Setup=
*Use [[Subversion]] to checkout Archivematica code
+
*Install ubuntu 12.04
**If you have commit access, use the private repository
+
*create a non-root user (with sudo privileges)
**If you have read-only access, use the public [http://code.google.com/p/archivematica/source/browse/trunk#trunk/includes/archivematica GoogleCode repository]
+
*log in as your new non-root user
<pre>svn checkout http://archivematica.googlecode.com/svn/trunk/ archivematica</pre>
+
*install archivematica storage service
 +
<pre>
 +
sudo apt-get update
 +
sudo apt-get install python-software-properties
 +
sudo add-apt-repository ppa:archivematica/release
 +
sudo add-apt-repository ppa:archivematica/externals
 +
sudo apt-get update
 +
sudo apt-get upgrade
 +
sudo apt-get install archivematica-storage-service
 +
sudo rm /etc/nginx/sites-enabled/default
 +
sudo ln -s /etc/nginx/sites-available/storage /etc/nginx/sites-enabled/storage
 +
sudo ln -s /etc/uwsgi/apps-available/storage.ini /etc/uwsgi/apps-enabled/storage.ini
 +
sudo service uwsgi restart
 +
sudo service nginx restart
 +
</pre>
  
 +
*install git
 +
<pre>sudo apt-get install git</pre>
  
*Run localDevSetup scripts.
+
*Use git to checkout Archivematica code
** <pre>cd archivematica/localDevSetup/</pre>
+
<pre>git clone https://github.com/artefactual/archivematica.git</pre>
** <pre>./installLocalDevRequirements.sh</pre>
+
** <pre>./createArchivematicaUserAndGroupForLocalDev.sh</pre>
+
** <pre>./createLocalDevDirectories.sh</pre>
+
** <pre>cd createDatabases</pre>
+
** <pre>./postBuildRun.sh</pre>
+
**This will create links in your local Ubuntu environment back to the relevant Archivematica scripts in your SVN checkout, allowing you to run a local Archivematica version which can continue to pull down revision updates from the code repository
+
**If you have commit access, you can work on the files in the checkout directory, make changes and commit those back to SVN repository, while also running Archivematica locally to test your changes and commits from other developers. 
+
  
*Additional local environment configuration
+
*Run the install and helper scripts.  
** Edit the sudoers file to give archivematica execution rights.
+
** <pre>cd archivematica</pre>
** <pre>sudo echo "archivematica ALL=NOPASSWD:/bin/mv,/bin/chown,/bin/chmod,/usr/bin/unoconv,/usr/bin/gs" >> /etc/sudoers </pre>
+
** <pre>./dev-installer</pre>
** Add Crontab for quarantine check
+
** Answer Y to all prompts.
** <pre>sudo crontab -e</pre>
+
** Restart the machine. (this is to enable the upstart services).
** <pre>* * * * * flock -xn /var/lock/quarantine.lock /usr/lib/archivematica/MCPServer/delay/delay.py 60 "/var/archivematica/sharedDirectory/watchedDirectories/quarantined"</pre>
+
** <pre>cd archivematica</pre>
 +
** <pre>./dev-helper</pre>
 +
** Answer Y to all prompts.
 +
** Complete AtoM setup
 +
*** The database should already be created.
 +
*** http://localhost/atom
 +
*** https://www.qubit-toolkit.org/wiki/Installation#Open_Qubit.2C_ICA-AtoM.2C_or_DCB_in_your_web_browser
 +
** Open the [dashboard http://localhost]
  
=Run=
+
=Configuring and Using Archivematica=
*Start Archivematica MCP Server & Client
+
These manuals contain information about the use and configuration of Archivematica:
**Open a seperate terminal for each and type:
+
<br/>https://www.archivematica.org/wiki/Administrator_manual
***<pre>runArchivematicaMCPServer.sh</pre>
+
<br/>https://www.archivematica.org/wiki/User_Manual
***<pre>runArchivematicaMCPClient.sh</pre>
+
 
+
*Start unoconv service (for Open Document format transcoding)
+
**<pre>unoconv --listen</pre>
+
 
+
*To start processing a SIP:
+
**Open Thunar file manager
+
**if using sample SIPs, 'export' the SIP directory
+
***<pre>svn export buildVM/includes/sampledata ~/sampledata</pre>
+
**'cut' the SIP directory and paste into the AcquireSIP directory
+
**Open the Firefox browser and type 'localhost' as the address to view SIP processing progress via the Dashboard
+
***note: until polling is enabled, you will have to hit the browser refresh button to get latest Dashboard updates
+
  
 
=Update=
 
=Update=
To pull down the latest code commits from the repository, navigate to the directory where Archivematica trunk has been checked out:
+
 
*<pre>svn up</pre>
+
To pull down the latest code commits from the repository, and reset your dev install, navigate to the directory where Archivematica has been cloned:
 +
*Change Directory to the archivematica git directory.
 +
<pre>cd ~/archivematica/</pre>
 +
*Check to see if you have any local changes that need to be stashed
 +
<pre>git diff</pre>
 +
*If there are local changes please stash or commit them. (Or you won't be able to update).
 +
<pre>git stash</pre>
 +
Update...
 +
*<pre>./dev-helper</pre>
 +
*<pre>"Would you like to git pull?" (y/N) y</pre>
 +
*<pre>"Would you like to update/install package requirements?" (y/N) </pre>
 +
*<pre>"Would you like to recreate the databases?" (y/N) y</pre>
 +
**Note: this command will remove any processing sips/transfers.
 +
*<pre>"Would you like to erase the ElasticSearch indexes?" (y/N) </pre>
 +
*<pre>"Would you like to clear transfer backlog and AIP storage?" (y/N) </pre>
 +
**Note: this command will delete all stored AIPs and stored backlogged transfers.
 +
*<pre>"Would you like to restart archivematica services?" (y/N) y</pre>
 +
*<pre>"Would you like to update sample data in /home/user/archivematica-sampledata?" (y/N) </pre>
 +
*<pre>"Would you like to export sample data from /home/user/archivematica-sampledata?" (y/N) </pre>
 +
*<pre>"Would you like to update AtoM and restart its atom-worker service?" (y/N) y</pre>
 +
If you stashed changes, re-apply them with
 +
<pre>git stash pop</pre>
  
 
=Troubleshooting=
 
=Troubleshooting=
 +
* If it stalls during update "Would you like to update/install package requirements?" (y/N)
 +
** Stop the script with CTRL + C
 +
** Try installing the item it failed on on the command line. Ie.
 +
** <pre>sudo apt-get install postfix</pre>
 +
** Restart the dev-helper "Would you like to update/install package requirements?"
 +
 +
* If you get this error error starting service:
 +
** <pre>sudo start archivematica-mcp-server</pre>
 +
** <pre>start: Unknown job: archivematica-mcp-server</pre>
 +
** Reload the configuration  [http://askubuntu.com/a/291644 Original answer] and try again
 +
** <pre>sudo initctl reload-configuration</pre>
 +
** Alternatively, reboot the machine.
 +
 
*If a SIP processing fails, it will move it to the 'failed' directory which is located:
 
*If a SIP processing fails, it will move it to the 'failed' directory which is located:
**<pre>var/archivematica/sharedDirectory/watchedDirectories/failed</pre>
+
**<pre>/var/archivematica/sharedDirectory/watchedDirectories/failed</pre>
  
*if ArchivematicaServer freezes (in terminal kill command)
+
*if ArchivematicaServer freezes  
**<pre>CTRL + \</pre>
+
**<pre>sudo restart archivematica-mcp-server</pre>
 +
**<pre>sudo restart archivematica-mcp-client</pre>
  
 
*if ArchivematicaClient freezes (in terminal kill command)
 
*if ArchivematicaClient freezes (in terminal kill command)
**<pre>CTRL + C</pre>
+
**<pre>sudo restart archivematica-mcp-client</pre>
  
 
*Updates to the Dashboard may require an Apache webserver restart:
 
*Updates to the Dashboard may require an Apache webserver restart:
 
**<pre>sudo /etc/init.d/apache2 restart</pre>
 
**<pre>sudo /etc/init.d/apache2 restart</pre>
  
*Updates to the environment may require re-linking:
+
*If you find a problem running the Dashboard and you want to get a detailed error log to report us, please switch it to debug mode following [http://archivematica.org/wiki/index.php?title=Dashboard#Debug_mode these instructions].
**<pre>cd archivematica/localDevSetup/</pre>
+
 
**<pre>./cleanup.sh</pre>
+
*MCP is currently logging to the /tmp/directory
**<pre>./createLocalDevDirectories.sh</pre>
+
** /tmp/archivematicaMCPClient-HOST.log
 +
** /tmp/archivematicaMCPServer-HOST-DATE.log
 +
** /tmp/archivematicaMCPServerPID
 +
 
 +
[[Category:Deprecated]]

Latest revision as of 14:23, 23 March 2017

Main Page > Development > Development Environment

Please note: Development environment instructions were for Archivematica 1.4 and earlier - these notes are no longer supported. Instead please see: Getting started

This page explains how you can configure and use a standard Linux system as an Archivematica development environment. The Archivematica development environment is available for developers that want the ability to customize or enhance their own Archivematica installation and/or contribute code back to the Archivematica project.

Contents

[edit] Setup

  • Install ubuntu 12.04
  • create a non-root user (with sudo privileges)
  • log in as your new non-root user
  • install archivematica storage service
sudo apt-get update
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:archivematica/release
sudo add-apt-repository ppa:archivematica/externals
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install archivematica-storage-service
sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/storage /etc/nginx/sites-enabled/storage
sudo ln -s /etc/uwsgi/apps-available/storage.ini /etc/uwsgi/apps-enabled/storage.ini
sudo service uwsgi restart
sudo service nginx restart
  • install git
sudo apt-get install git
  • Use git to checkout Archivematica code
git clone https://github.com/artefactual/archivematica.git

[edit] Configuring and Using Archivematica

These manuals contain information about the use and configuration of Archivematica:
https://www.archivematica.org/wiki/Administrator_manual
https://www.archivematica.org/wiki/User_Manual

[edit] Update

To pull down the latest code commits from the repository, and reset your dev install, navigate to the directory where Archivematica has been cloned:

  • Change Directory to the archivematica git directory.
cd ~/archivematica/
  • Check to see if you have any local changes that need to be stashed
git diff
  • If there are local changes please stash or commit them. (Or you won't be able to update).
git stash

Update...

  • ./dev-helper
  • "Would you like to git pull?" (y/N) y
  • "Would you like to update/install package requirements?" (y/N) 
  • "Would you like to recreate the databases?" (y/N) y
    • Note: this command will remove any processing sips/transfers.
  • "Would you like to erase the ElasticSearch indexes?" (y/N) 
  • "Would you like to clear transfer backlog and AIP storage?" (y/N) 
    • Note: this command will delete all stored AIPs and stored backlogged transfers.
  • "Would you like to restart archivematica services?" (y/N) y
  • "Would you like to update sample data in /home/user/archivematica-sampledata?" (y/N) 
  • "Would you like to export sample data from /home/user/archivematica-sampledata?" (y/N) 
  • "Would you like to update AtoM and restart its atom-worker service?" (y/N) y

If you stashed changes, re-apply them with

git stash pop

[edit] Troubleshooting

  • If it stalls during update "Would you like to update/install package requirements?" (y/N)
    • Stop the script with CTRL + C
    • Try installing the item it failed on on the command line. Ie.
    • sudo apt-get install postfix
    • Restart the dev-helper "Would you like to update/install package requirements?"
  • If you get this error error starting service:
    • sudo start archivematica-mcp-server
    • start: Unknown job: archivematica-mcp-server
    • Reload the configuration Original answer and try again
    • sudo initctl reload-configuration
    • Alternatively, reboot the machine.
  • If a SIP processing fails, it will move it to the 'failed' directory which is located:
    • /var/archivematica/sharedDirectory/watchedDirectories/failed
  • if ArchivematicaServer freezes
    • sudo restart archivematica-mcp-server
    • sudo restart archivematica-mcp-client
  • if ArchivematicaClient freezes (in terminal kill command)
    • sudo restart archivematica-mcp-client
  • Updates to the Dashboard may require an Apache webserver restart:
    • sudo /etc/init.d/apache2 restart
  • If you find a problem running the Dashboard and you want to get a detailed error log to report us, please switch it to debug mode following these instructions.
  • MCP is currently logging to the /tmp/directory
    • /tmp/archivematicaMCPClient-HOST.log
    • /tmp/archivematicaMCPServer-HOST-DATE.log
    • /tmp/archivematicaMCPServerPID
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox