Code Repositories xandikos / upstream/0.0.7
New upstream version 0.0.7 Jelmer Vernooij 1 year, 5 months ago
48 changed file(s) with 2244 addition(s) and 1124 deletion(s). Raw diff Collapse all Expand all
1212 - pip install pip --upgrade
1313 - pip install coverage codecov flake8 pycalendar
1414 - sudo apt-get install -qq libneon27-dev curl python2.7
15 - sudo apt-get install -qq cargo
1516 - python setup.py develop
1617 script:
1718 - make style
1819 - make coverage
1920 - mv .coverage .coverage.unit
20 - make coverage-litmus
21 # Retrieve litmus from Xandikos server for now, since webdav.org is down.
22 - make coverage-litmus LITMUS_URL=https://www.xandikos.org/litmus-0.13.tar.gz
2123 - mv .coverage .coverage.litmus
22 - make coverage-vdirsyncer
23 - mv .coverage .coverage.vdirsyncer
24 - if [ "$TRAVIS_PYTHON_VERSION" = "3.6" ]; then
25 make coverage-vdirsyncer;
26 mv .coverage .coverage.vdirsyncer;
27 fi
2428 - make coverage-caldavtester
2529 - mv .coverage .coverage.caldavtester
2630 after_success:
2731 - python -m coverage combine
2832 - codecov
33 cache:
34 pip: true
0 FROM debian:sid
1 LABEL maintainer="jelmer@jelmer.uk"
2 RUN apt-get update && \
3 apt-get -y install uwsgi uwsgi-plugin-python3 python3-icalendar python3-dulwich python3-jinja2 python3-defusedxml && \
4 apt-get clean
5 ADD . /code
6 WORKDIR /code
7 VOLUME /data
8 EXPOSE 8000
9 ENV autocreate="defaults"
10 ENV current_user_principal="/dav/user1"
11
12 # TODO(jelmer): Add support for authentication
13 # --plugin=router_basicauth,python3 --route="^/ basicauth:myrealm,user1:password1"
14 CMD uwsgi --http-socket=:8000 --umask=022 --master --cheaper=2 --processes=4 --plugin=python3 --module=xandikos.wsgi:app --env=XANDIKOSPATH=/data --env=CURRENT_USER_PRINCIPAL=$current_user_principal --env=AUTOCREATE=$autocreate
00 Metadata-Version: 1.1
11 Name: xandikos
2 Version: 0.0.6
2 Version: 0.0.7
33 Summary: Lightweight CalDAV/CardDAV server
44 Home-page: https://www.xandikos.org/
55 Author: Jelmer Vernooij
66 Author-email: jelmer@jelmer.uk
77 License: GNU GPLv3 or later
8 Description-Content-Type: UNKNOWN
89 Description: UNKNOWN
910 Platform: UNKNOWN
1011 Classifier: Development Status :: 4 - Beta
5959 - `CalDAV-Sync <https://dmfs.org/caldav/>`_
6060 - `CardDAV-Sync <https://dmfs.org/carddav/>`_
6161 - `Calendarsync <https://play.google.com/store/apps/details?id=com.icalparse>`_
62 - `Tasks <https://github.com/tasks/tasks/tree/caldav>`_
63 - `AgendaV <http://agendav.org/>`_
64
65 Client instructions
66 ===================
67
68 Some clients can automatically discover the calendar and addressbook URLs from
69 a DAV server. For such clients you can simply provide the URL to Xandikos directly.
70
71 Clients that lack such automated discovery require the direct URL to a calendar
72 or addressbook. One such client is Thunderbird lightning in which case you
73 should provide a URL similar to the following:
74
75 ::
76
77 http://dav.example.com/user/calendars/my_calendar
6278
6379 Dependencies
6480 ============
77 TEST_ARG=TESTS="$TESTS"
88 fi
99 SRCPATH="$(dirname $(readlink -m $0))"
10 VERSION=0.13
10 VERSION=${LITMUS_VERSION:-0.13}
11 LITMUS_URL="${LITMUS_URL:-http://www.webdav.org/neon/litmus/litmus-${VERSION}.tar.gz}"
1112
1213 scratch=$(mktemp -d)
1314 function finish() {
1920 if [ -f "${SRCPATH}/litmus-${VERSION}.tar.gz" ]; then
2021 cp "${SRCPATH}/litmus-${VERSION}.tar.gz" .
2122 else
22 wget -O "litmus-${VERSION}.tar.gz" http://www.webdav.org/neon/litmus/litmus-${VERSION}.tar.gz
23 wget -O "litmus-${VERSION}.tar.gz" "${LITMUS_URL}"
2324 fi
2425 sha256sum ${SRCPATH}/litmus-${VERSION}.tar.gz.sha256sum
2526 tar xvfz litmus-${VERSION}.tar.gz
2121 export PYTHONPATH=${REPO_DIR}
2222 pushd ${REPO_DIR} && ${PYTHON} setup.py develop && popd
2323 fi
24
25 if [ -z "${CARGO_HOME}" ]; then
26 export CARGO_HOME="$(readlink -f .)/cargo"
27 export RUSTUP_HOME="$(readlink -f .)/cargo"
28 fi
29 curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain nightly --no-modify-path
30 . ${CARGO_HOME}/env
31 rustup update nightly
32
33 # Add --ignore=tests/system/utils/test_main.py since it fails in travis,
34 # and isn't testing anything relevant to Xandikos.
2435 make \
2536 COVERAGE=true \
26 PYTEST_ARGS="${PYTEST_ARGS} tests/storage/dav/" \
37 PYTEST_ARGS="${PYTEST_ARGS} tests/storage/dav/ --ignore=tests/system/utils/test_main.py" \
2738 DAV_SERVER=xandikos \
2839 install-dev install-test test
0 [uwsgi]
1 http-socket = :$(PORT)
2 die-on-term = true
3 umask = 022
4 master = true
5 cheaper = 0
6 processes = 1
7 plugin = router_basicauth,python3
8 route = ^/ basicauth:myrealm,user1:password1
9 module = xandikos.wsgi:app
10 env = XANDIKOSPATH=$HOME/dav
11 env = CURRENT_USER_PRINCIPAL=/dav/user1/
12 env = AUTOCREATE=defaults
11 http-socket = 127.0.0.1:8080
22 umask = 022
33 master = true
4 cheaper = 2
5 processes = 4
4 cheaper = 0
5 processes = 1
66 plugin = router_basicauth,python3
77 route = ^/ basicauth:myrealm,user1:password1
88 module = xandikos.wsgi:app
22 uid = xandikos
33 gid = xandikos
44 master = true
5 cheaper = 2
6 processes = 4
5 cheaper = 0
6 processes = 1
77 plugin = python3
88 module = xandikos.wsgi:app
99 umask = 022
0 # This an example .xandikos file.
1
2 # The color for this collection is red
3 color = FF0000
4
5 inbox-url = inbox/
0 API Stability
1 =============
2
3 There are currently no guarantees about Xandikos Python APIs staying the same
4 across different versions, except the following APIs:
5
6 xandikos.web.XandikosBackend(path)
7 xandikos.web.XandikosBackend.create_principal(principal, create_defaults=False)
8 xandikos.web.XandikosApp(backend, current_user_principal)
9 xandikos.web.WellknownRedirector(app, path)
10
11 If you care about stability of any other APIs, please file a bug against Xandikos.
0 Per-collection configuration
1 ============================
2
3 Xandikos needs to store several piece of per-collection metadata.
4
5 Some of these can be inferred from other sources.
6
7 For starters, for each collection:
8
9 - resource types: principal, calendar, addressbook
10
11 Per resource type-specific properties
12 -------------------------------------
13
14 Principal
15 ~~~~~~~~~
16
17 Per principal configuration settings:
18
19 - calendar home sets
20 - addressbook home sets
21 - user address set
22 - infit settings
23
24 Calendar
25 ~~~~~~~~
26
27 Need per calendar config:
28
29 - color
30 - description (can be inferred from .git/description)
31 - inbox URL
32 - outbox URL
33 - max instances
34 - max attendees per instance
35 - calendar timezone
36 - calendar schedule transparency
37
38 Addressbook
39 ~~~~~~~~~~~
40
41 Need per addressbook config:
42
43 - max image size
44 - max resource size
45 - color
46 - description (can be inferred from .git/description)
47
48 Schedule Inbox
49 ~~~~~~~~~~~~~~
50 - default-calendar-URL
51
52 Proposed format
53 ---------------
54
55 Store a ini-style .xandikos file in the directory hosting the Collection (or
56 Tree in case of a Git repository).
57
58 Example
59 -------
60 # This is a standard Python configobj file, so it's mostly ini-style, and comments
61 # can appear preceded by #.
62
63 color = 030003
64
65
+0
-32
notes/config.rst less more
0 Principal
1 =========
2
3 Need per principal config:
4
5 - calendar home sets
6 - addressbook home sets
7 - user address set
8 - infit settings
9
10 Calendar
11 ========
12
13 Need per calendar config:
14
15 - color
16 - description
17 - inbox URL
18 - outbox URL
19 - max instances
20 - max attendees per instance
21 - calendar timezone
22
23 Addresssbook
24 ============
25
26 Need per addressbook config:
27
28 - max image size
29 - max resource size
30 - color
31 - description
189189 - CARDDAV:addressbook-query [supported]
190190 - CARDDAV:addressbook-multiget [supported]
191191
192 rfc6638.txt (CardDAV scheduling extensions)
193 -------------------------------------------
192 rfc6638.txt (CalDAV scheduling extensions)
193 ------------------------------------------
194194
195195 DAV Properties
196196 ^^^^^^^^^^^^^^
199199 - CALDAV:schedule-inbox-URL [supported]
200200 - CALDAV:calendar-user-address-set [supported]
201201 - CALDAV:calendar-user-type [supported]
202 - CALDAV:schedule-calendar-transp [supported]
203 - CALDAV:schedule-default-calendar-URL [supported]
204 - CALDAV:schedule-tag [not supported]
202205
203206 rfc6764.txt (Locating groupware services)
204207 -----------------------------------------
247250
248251 - calendar-color [supported]
249252 - getctag [supported]
253 - refreshrate [supported]
250254
251255 inf-it properties
252256 ^^^^^^^^^^^^^^^^^
254258 - headervalue [supported]
255259 - settings [supported]
256260 - addressbook-color [supported]
261
262 AgendaV properties
263 ^^^^^^^^^^^^^^^^^^
264
265 https://tools.ietf.org/id/draft-ietf-calext-caldav-attachments-03.html
266
267 - CALDAV:max-attachments-per-resource [supported]
268 - CALDAV:max-attachment-size [supported]
269 - CALDAV:managed-attachments-server-URL [supported]
257270
258271 rfc5995.txt (POST to create members)
259272 ------------------------------------
0 Running Xandikos on Heroku
1 ==========================
2
3 Heroku is an easy way to get a public instance of Xandikos running. A free
4 heroku instance comes with 100Mb of local storage, which is enough for
5 thousands of calendar items or contacts.
6
7 Deployment
8 ----------
9
10 All of these steps assume you already have a Heroku account and have installed
11 the heroku command-line client.
12
13 To run a Heroku instance with Xandikos:
14
15 1. Create a copy of Xandikos::
16
17 $ git clone git://jelmer.uk/xandikos xandikos
18 $ cd xandikos
19
20 2. Make a copy of the example uwsgi configuration::
21
22 $ cp examples/uwsgi-heroku.ini uwsgi.ini
23
24 3. Edit *uwsgi.ini* as necessary, such as changing the credentials (the
25 defaults are *user1*/*password1*).
26
27 4. Make heroku install and use uwsgi::
28
29 $ echo uwsgi > requirements.txt
30 $ echo web: uwsgi uwsgi.ini > Procfile
31
32 5. Create the Heroku instance::
33
34 $ heroku create
35
36 (this might ask you for your heroku credentials)
37
38 6. Deploy the app::
39
40 $ git push heroku master
41
42 7. Open the app with your browser::
43
44 $ heroku open
45
46 (The URL opened is also the URL that you can provide to any CalDAV/CardDAV
47 application that supports service discovery)
0 Multi-User Support
1 ==================
2
3 Multi-user support could arguably also include sharing of
4 calendars/collections/etc. This is beyond the scope of this document, which
5 just focuses on allowing multiple users to use their own silo in a single
6 instance of Xandikos.
7
8 Siloed user support can be split up into three steps:
9
10 * storage - mapping a user to a principal
11 * authentication - letting a user log in
12 * authorization - checking whether the user has access to a resource
13
14 Authentication
15 --------------
16
17 In the simplest form, a forwarding proxy provides the name of an authenticated
18 user. E.g. Apache or uWSGI sets the REMOTE_USER environment variable. If
19 REMOTE_USER is not present for an operation that requires authentication, a 401
20 error is returned.
21
22 Authorization
23 -------------
24
25 In the simplest form, users only have access to the resources under their own
26 principal.
27
28 Storage
29 -------
30
31 By default, the principal for a user is simply "/%(username)s".
32
33 Roadmap
34 =======
35
36 * Allow marking collections as principals
37 * Expose username (or None, if not logged in) everywhere
38 * Add function get_username_principal() for mapping username to principal path
39 * Add simple function check_path_access() for checking access ("is this user allowed to access this path?")
40 * Use access checking function everywhere
41 * Have current-user-principal setting depend on $REMOTE_USER and get_username_principal()
0 CalDAV Scheduling
1 =================
2
3 TODO:
4
5 - When a new calendar object is uploaded to a calendar collection:
6 * Check if the ATTENDEE property is present, and if so, process it
7
8 - Support CALDAV:schedule-tag
9 * When comparing with if-schedule-tag-match, simply retrieve the blob by schedule-tag and compare delta between newly uploaded and current
10 * When determining schedule-tag, scroll back until last revision that didn't have attendee changes?
11 + Perhaps include a hint in e.g. commit message?
12
13 - Inbox "contains copies of incoming scheduling messages"
14 - Outbox "at which busy time information requests are targeted."
1919 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
2020 # MA 02110-1301, USA.
2121
22 from setuptools import setup
22 from setuptools import find_packages, setup
23 import sys
2324
24 version = "0.0.6"
25 version = "0.0.7"
26
27 if sys.platform != 'win32':
28 # Win32 setup breaks on non-ascii characters
29 author = "Jelmer Vernooij"
30 else:
31 author = "Jelmer Vernooij"
2532
2633 setup(name="xandikos",
2734 description="Lightweight CalDAV/CardDAV server",
2835 version=version,
29 author="Jelmer Vernooij",
36 author=author,
3037 author_email="jelmer@jelmer.uk",
3138 license="GNU GPLv3 or later",
3239 url="https://www.xandikos.org/",
3340 install_requires=['icalendar', 'dulwich', 'defusedxml', 'jinja2'],
34 packages=['xandikos', 'xandikos.tests'],
41 packages=find_packages(),
3542 package_data={'xandikos': ['templates/*.html']},
3643 scripts=['bin/xandikos'],
3744 test_suite='xandikos.tests.test_suite',
2020
2121 """CalDAV/CardDAV server."""
2222
23 __version__ = (0, 0, 6)
23 __version__ = (0, 0, 7)
24 version_string = '.'.join(map(str, __version__))
2425
2526 import defusedxml.ElementTree # noqa: This does some monkey-patching on-load
3939 in_allprops = False
4040 live = True
4141
42 def get_value(self, href, resource, el):
42 def get_value(self, href, resource, el, environ):
4343 privilege = ET.SubElement(el, '{DAV:}privilege')
4444 # TODO(jelmer): Use something other than all
4545 ET.SubElement(privilege, '{DAV:}all')
5555 in_allprops = False
5656 live = True
5757
58 def get_value(self, base_href, resource, el):
58 def get_value(self, base_href, resource, el, environ):
5959 owner_href = resource.get_owner()
6060 if owner_href is not None:
6161 el.append(webdav.create_href(owner_href, base_href=base_href))
7272 live = True
7373 resource_type = webdav.PRINCIPAL_RESOURCE_TYPE
7474
75 def get_value(self, base_href, resource, el):
75 def get_value(self, base_href, resource, el, environ):
7676 for href in resource.get_group_membership():
7777 el.append(webdav.create_href(href, base_href=href))
3333 resource_type = None
3434 live = False
3535
36 def get_value(self, href, resource, el):
36 def get_value(self, href, resource, el, environ):
3737 el.text = ('T' if resource.get_is_executable() else 'F')
3838
3939 def set_value(self, href, resource, el):
3535 WELLKNOWN_CALDAV_PATH = "/.well-known/caldav"
3636 EXTENDED_MKCOL_FEATURE = 'extended-mkcol'
3737
38 NAMESPACE = 'urn:ietf:params:xml:ns:caldav'
39
3840 # https://tools.ietf.org/html/rfc4791, section 4.2
39 CALENDAR_RESOURCE_TYPE = '{urn:ietf:params:xml:ns:caldav}calendar'
40
41 NAMESPACE = 'urn:ietf:params:xml:ns:caldav'
41 CALENDAR_RESOURCE_TYPE = '{%s}calendar' % NAMESPACE
42
43 # TODO(jelmer): These resource types belong in scheduling.py
44 SCHEDULE_INBOX_RESOURCE_TYPE = '{%s}schedule-inbox' % NAMESPACE
45 SCHEDULE_OUTBOX_RESOURCE_TYPE = '{%s}schedule-outbox' % NAMESPACE
4246
4347 # Feature to advertise to indicate CalDAV support.
4448 FEATURE = 'calendar-access'
49
50 TRANSPARENCY_TRANSPARENT = 'transparent'
51 TRANSPARENCY_OPAQUE = 'opaque'
52
53
54 class MissingProperty(Exception):
55
56 def __init__(self, property_name):
57 super(MissingProperty, self).__init__(
58 "Property %r missing" % property_name)
59 self.property_name = property_name
4560
4661
4762 class Calendar(webdav.Collection):
99114 def get_max_date_time(self):
100115 """Return maximum datetime property.
101116 """
102 raise NotImplementedError(self.get_min_date_time)
117 raise NotImplementedError(self.get_max_date_time)
103118
104119 def get_max_instances(self):
105120 """Return maximum number of instances.
111126 """
112127 raise NotImplementedError(self.get_max_attendees_per_instance)
113128
129 def get_max_resource_size(self):
130 """Return max resource size."""
131 raise NotImplementedError(self.get_max_resource_size)
132
133 def get_max_attachments_per_resource(self):
134 """Return max attachments per resource."""
135 raise NotImplementedError(self.get_max_attachments_per_resource)
136
137 def get_max_attachment_size(self):
138 """Return max attachment size."""
139 raise NotImplementedError(self.get_max_attachment_size)
140
141 def get_managed_attachments_server_url(self):
142 """Return the attachments server URL."""
143 raise NotImplementedError(self.get_managed_attachments_server_url)
144
145 def get_schedule_calendar_transparency(self):
146 """Get calendar transparency.
147
148 Possible values are TRANSPARENCY_TRANSPARENT and TRANSPARENCY_OPAQUE
149 """
150 return TRANSPARENCY_OPAQUE
151
114152
115153 class PrincipalExtensions:
116154 """CalDAV-specific extensions to DAVPrincipal."""
136174 See https://www.ietf.org/rfc/rfc4791.txt, section 6.2.1.
137175 """
138176
139 name = '{urn:ietf:params:xml:ns:caldav}calendar-home-set'
177 name = '{%s}calendar-home-set' % NAMESPACE
140178 resource_type = '{DAV:}principal'
141179 in_allprops = False
142180 live = True
143181
144 def get_value(self, base_href, resource, el):
182 def get_value(self, base_href, resource, el, environ):
145183 for href in resource.get_calendar_home_set():
146184 href = webdav.ensure_trailing_slash(href)
147185 el.append(webdav.create_href(href, base_href))
153191 https://tools.ietf.org/html/rfc4791, section 5.2.1
154192 """
155193
156 name = '{urn:ietf:params:xml:ns:caldav}calendar-description'
194 name = '{%s}calendar-description' % NAMESPACE
157195 resource_type = CALENDAR_RESOURCE_TYPE
158196
159 def get_value(self, base_href, resource, el):
197 def get_value(self, base_href, resource, el, environ):
160198 el.text = resource.get_calendar_description()
161199
162200 # TODO(jelmer): allow modification of this property
200238 def supported_on(self, resource):
201239 return (resource.get_content_type() == 'text/calendar')
202240
203 def get_value_ext(self, base_href, resource, el, requested):
241 def get_value_ext(self, base_href, resource, el, environ, requested):
204242 if len(requested) == 0:
205243 serialized_cal = b''.join(resource.get_body())
206244 else:
211249 extract_from_calendar(calendar, c, requested)
212250 serialized_cal = c.to_ical()
213251 # TODO(jelmer): Don't hardcode encoding
252 # TODO(jelmer): Strip invalid characters or raise an exception
214253 el.text = serialized_cal.decode('utf-8')
215254
216255
217256 class CalendarMultiGetReporter(davcommon.MultiGetReporter):
218257
219 name = '{urn:ietf:params:xml:ns:caldav}calendar-multiget'
220 resource_type = CALENDAR_RESOURCE_TYPE
258 name = '{%s}calendar-multiget' % NAMESPACE
259 resource_type = (CALENDAR_RESOURCE_TYPE, SCHEDULE_INBOX_RESOURCE_TYPE)
221260 data_property = CalendarDataProperty()
222261
223262
256295 def apply_text_match(el, value):
257296 collation = el.get('collation', 'i;ascii-casemap')
258297 negate_condition = el.get('negate-condition', 'no')
259 matches = davcommon.collations[collation](el.text, value)
298 matches = davcommon.get_collation(collation)(el.text, value)
260299
261300 if negate_condition == 'yes':
262301 return (not matches)
314353
315354
316355 def apply_time_range_vevent(start, end, comp, tzify):
356 if comp['DTSTART'] is None:
357 raise MissingProperty('DTSTART')
358
317359 if not (end > tzify(comp['DTSTART'].dt)):
318360 return False
319361
508550
509551 class CalendarQueryReporter(webdav.Reporter):
510552
511 name = '{urn:ietf:params:xml:ns:caldav}calendar-query'
512 resource_type = CALENDAR_RESOURCE_TYPE
553 name = '{%s}calendar-query' % NAMESPACE
554 resource_type = (CALENDAR_RESOURCE_TYPE, SCHEDULE_INBOX_RESOURCE_TYPE)
513555 data_property = CalendarDataProperty()
514556
515557 @webdav.multistatus
536578 tzify = lambda dt: as_tz_aware_ts(dt, tz)
537579 for (href, resource) in webdav.traverse_resource(
538580 base_resource, base_href, depth):
539 if not apply_filter(filter_el, resource, tzify):
581 try:
582 filter_result = apply_filter(filter_el, resource, tzify)
583 except MissingProperty as e:
584 logging.warning(
585 'calendar_query: Ignoring calendar object %s, due '
586 'to missing property %s', href, e.property_name)
587 continue
588 if not filter_result:
540589 continue
541590 propstat = davcommon.get_properties_with_data(
542 self.data_property, href, resource, properties, requested)
591 self.data_property, href, resource, properties, environ,
592 requested)
543593 yield webdav.Status(href, '200 OK', propstat=list(propstat))
544594
545595
552602 name = '{http://apple.com/ns/ical/}calendar-color'
553603 resource_type = CALENDAR_RESOURCE_TYPE
554604
555 def get_value(self, href, resource, el):
605 def get_value(self, href, resource, el, environ):
556606 el.text = resource.get_calendar_color()
557607
558608 def set_value(self, href, resource, el):
567617 See https://www.ietf.org/rfc/rfc4791.txt, section 5.2.3
568618 """
569619
570 name = '{urn:ietf:params:xml:ns:caldav}supported-calendar-component-set'
571 resource_type = CALENDAR_RESOURCE_TYPE
572 in_allprops = False
573 live = True
574
575 def get_value(self, href, resource, el):
620 name = '{%s}supported-calendar-component-set' % NAMESPACE
621 resource_type = (CALENDAR_RESOURCE_TYPE,
622 SCHEDULE_INBOX_RESOURCE_TYPE,
623 SCHEDULE_OUTBOX_RESOURCE_TYPE)
624 in_allprops = False
625 live = True
626
627 def get_value(self, href, resource, el, environ):
576628 for component in resource.get_supported_calendar_components():
577629 subel = ET.SubElement(el, '{urn:ietf:params:xml:ns:caldav}comp')
578630 subel.set('name', component)
585637 """
586638
587639 name = '{urn:ietf:params:xml:ns:caldav}supported-calendar-data'
588 resource_type = CALENDAR_RESOURCE_TYPE
589 in_allprops = False
590
591 def get_value(self, href, resource, el):
640 resource_type = (CALENDAR_RESOURCE_TYPE,
641 SCHEDULE_INBOX_RESOURCE_TYPE,
642 SCHEDULE_OUTBOX_RESOURCE_TYPE)
643 in_allprops = False
644
645 def get_value(self, href, resource, el, environ):
592646 for (content_type, version) in (
593647 resource.get_supported_calendar_data_types()):
594648 subel = ET.SubElement(
604658 """
605659
606660 name = '{urn:ietf:params:xml:ns:caldav}calendar-timezone'
607 resource_type = CALENDAR_RESOURCE_TYPE
608 in_allprops = False
609
610 def get_value(self, href, resource, el):
661 resource_type = (CALENDAR_RESOURCE_TYPE,
662 SCHEDULE_INBOX_RESOURCE_TYPE)
663 in_allprops = False
664
665 def get_value(self, href, resource, el, environ):
611666 el.text = resource.get_calendar_timezone()
612667
613668 def set_value(self, href, resource, el):
624679 """
625680
626681 name = '{urn:ietf:params:xml:ns:caldav}min-date-time'
627 resource_type = CALENDAR_RESOURCE_TYPE
628 in_allprops = False
629 live = True
630
631 def get_value(self, href, resource, el):
682 resource_type = (CALENDAR_RESOURCE_TYPE,
683 SCHEDULE_INBOX_RESOURCE_TYPE,
684 SCHEDULE_OUTBOX_RESOURCE_TYPE)
685 in_allprops = False
686 live = True
687
688 def get_value(self, href, resource, el, environ):
632689 el.text = resource.get_min_date_time()
633690
634691
639696 """
640697
641698 name = '{urn:ietf:params:xml:ns:caldav}max-date-time'
642 resource_type = CALENDAR_RESOURCE_TYPE
643 in_allprops = False
644 live = True
645
646 def get_value(self, href, resource, el):
699 resource_type = (CALENDAR_RESOURCE_TYPE,
700 SCHEDULE_INBOX_RESOURCE_TYPE,
701 SCHEDULE_OUTBOX_RESOURCE_TYPE)
702 in_allprops = False
703 live = True
704
705 def get_value(self, href, resource, el, environ):
647706 el.text = resource.get_max_date_time()
648707
649708
653712 See https://tools.ietf.org/html/rfc4791, section 5.2.8
654713 """
655714
656 name = '{urn:ietf:params:xml:ns:caldav}max-instances'
657 resource_type = CALENDAR_RESOURCE_TYPE
658 in_allprops = False
659 live = True
660
661 def get_value(self, href, resource, el):
715 name = '{%s}max-instances' % NAMESPACE
716 resource_type = (CALENDAR_RESOURCE_TYPE,
717 SCHEDULE_INBOX_RESOURCE_TYPE)
718 in_allprops = False
719 live = True
720
721 def get_value(self, href, resource, el, environ):
662722 el.text = str(resource.get_max_instances())
663723
664724
668728 See https://tools.ietf.org/html/rfc4791, section 5.2.9
669729 """
670730
671 name = '{urn:ietf:params:xml:ns:caldav}max-attendees-per-instance'
731 name = '{%s}max-attendees-per-instance' % NAMESPACE
732 resource_type = (CALENDAR_RESOURCE_TYPE,
733 SCHEDULE_INBOX_RESOURCE_TYPE,
734 SCHEDULE_OUTBOX_RESOURCE_TYPE)
735 in_allprops = False
736 live = True
737
738 def get_value(self, href, resource, el, environ):
739 el.text = str(resource.get_max_attendees_per_instance())
740
741
742 class MaxResourceSizeProperty(webdav.Property):
743 """max-resource-size property.
744
745 See https://tools.ietf.org/html/rfc4791, section 5.2.5
746 """
747
748 name = '{%s}max-resource-size' % NAMESPACE
749 resource_type = (CALENDAR_RESOURCE_TYPE,
750 SCHEDULE_INBOX_RESOURCE_TYPE,
751 SCHEDULE_OUTBOX_RESOURCE_TYPE)
752 in_allprops = False
753 live = True
754
755 def get_value(self, href, resource, el, environ):
756 el.text = str(resource.get_max_resource_size())
757
758
759 class MaxAttachmentsPerResourceProperty(webdav.Property):
760 """max-attachments-per-resource property.
761
762 https://tools.ietf.org/id/draft-ietf-calext-caldav-attachments-03.html#rfc.section.6.3
763 """
764
765 name = '{%s}max-attachments-per-resource' % NAMESPACE
672766 resource_type = CALENDAR_RESOURCE_TYPE
673767 in_allprops = False
674768 live = True
675769
676 def get_value(self, href, resource, el):
677 el.text = str(resource.get_max_attendees_per_instance())
770 def get_value(self, href, resource, el, environ):
771 el.text = str(resource.get_max_attachments_per_resource())
772
773
774 class MaxAttachmentSizeProperty(webdav.Property):
775 """max-attachment-size property.
776
777 https://tools.ietf.org/id/draft-ietf-calext-caldav-attachments-03.html#rfc.section.6.2
778 """
779
780 name = '{%s}max-attachment-size' % NAMESPACE
781 resource_type = CALENDAR_RESOURCE_TYPE
782 in_allprops = False
783 live = True
784
785 def get_value(self, href, resource, el, environ):
786 el.text = str(resource.get_max_attachment_size())
787
788
789 class ManagedAttachmentsServerURLProperty(webdav.Property):
790 """managed-attachments-server-URL property.
791
792 https://tools.ietf.org/id/draft-ietf-calext-caldav-attachments-03.html#rfc.section.6.1
793 """
794
795 name = '{%s}managed-attachments-server-URL' % NAMESPACE
796 in_allprops = False
797
798 def get_value(self, base_href, resource, el, environ):
799 href = resource.get_managed_attachments_server_url()
800 if href is not None:
801 el.append(webdav.create_href(href, base_href))
678802
679803
680804 class CalendarProxyReadForProperty(webdav.Property):
689813 in_allprops = False
690814 live = True
691815
692 def get_value(self, base_href, resource, el):
816 def get_value(self, base_href, resource, el, environ):
693817 for href in resource.get_calendar_proxy_read_for():
694818 el.append(webdav.create_href(href, base_href))
695819
706830 in_allprops = False
707831 live = True
708832
709 def get_value(self, base_href, resource, el):
833 def get_value(self, base_href, resource, el, environ):
710834 for href in resource.get_calendar_proxy_write_for():
711835 el.append(webdav.create_href(href, base_href))
836
837
838 class ScheduleCalendarTransparencyProperty(webdav.Property):
839 """schedule-calendar-transp property.
840
841 See https://tools.ietf.org/html/rfc6638#section-9.1
842 """
843 name = '{%s}schedule-calendar-transp' % NAMESPACE
844 in_allprops = False
845 live = False
846 resource_type = CALENDAR_RESOURCE_TYPE
847
848 def get_value(self, base_href, resource, el, environ):
849 transp = resource.get_schedule_calendar_transparency()
850 if transp == TRANSPARENCY_TRANSPARENT:
851 ET.SubElement(el, '{%s}transparent' % NAMESPACE)
852 elif transp == TRANSPARENCY_OPAQUE:
853 ET.SubElement(el, '{%s}opaque' % NAMESPACE)
854 else:
855 raise ValueError('Invalid transparency %s' % transp)
712856
713857
714858 def map_freebusy(comp):
820964 start_response('409 Conflict', [])
821965 return []
822966 el = ET.Element('{DAV:}resourcetype')
823 app.properties['{DAV:}resourcetype'].get_value(href, resource, el)
967 app.properties['{DAV:}resourcetype'].get_value(
968 href, resource, el, environ)
824969 ET.SubElement(el, '{urn:ietf:params:xml:ns:caldav}calendar')
825970 app.properties['{DAV:}resourcetype'].set_value(href, resource, el)
826971 if base_content_type in ('text/xml', 'application/xml'):
4444 in_allprops = False
4545 live = True
4646
47 def get_value(self, base_href, resource, el):
47 def get_value(self, base_href, resource, el, environ):
4848 for href in resource.get_addressbook_home_set():
4949 href = webdav.ensure_trailing_slash(href)
5050 el.append(webdav.create_href(href, base_href))
6464 def supported_on(self, resource):
6565 return (resource.get_content_type() == 'text/vcard')
6666
67 def get_value_ext(self, href, resource, el, requested):
67 def get_value_ext(self, href, resource, el, environ, requested):
6868 # TODO(jelmer): Support subproperties
6969 # TODO(jelmer): Don't hardcode encoding
7070 el.text = b''.join(resource.get_body()).decode('utf-8')
7979 name = '{%s}addressbook-description' % NAMESPACE
8080 resource_type = ADDRESSBOOK_RESOURCE_TYPE
8181
82 def get_value(self, href, resource, el):
82 def get_value(self, href, resource, el, environ):
8383 el.text = resource.get_addressbook_description()
8484
8585 def set_value(self, href, resource, el):
157157 resource_type = '{DAV:}principal'
158158 in_allprops = False
159159
160 def get_value(self, href, resource, el):
160 def get_value(self, href, resource, el, environ):
161161 el.append(webdav.create_href(
162162 resource.get_principal_address(), href))
163163
173173 in_allprops = False
174174 live = True
175175
176 def get_value(self, href, resource, el):
176 def get_value(self, href, resource, el, environ):
177177 for (content_type,
178178 version) in resource.get_supported_address_data_types():
179179 subel = ET.SubElement(el, '{%s}content-type' % NAMESPACE)
192192 in_allprops = False
193193 live = True
194194
195 def get_value(self, href, resource, el):
195 def get_value(self, href, resource, el, environ):
196196 el.text = str(resource.get_max_resource_size())
197197
198198
207207 in_allprops = False
208208 live = True
209209
210 def get_value(self, href, resource, el):
210 def get_value(self, href, resource, el, environ):
211211 el.text = str(resource.get_max_image_size())
212212
213213
345345 if nresults is not None and i >= nresults:
346346 break
347347 propstat = davcommon.get_properties_with_data(
348 self.data_property, href, resource, properties, requested)
348 self.data_property, href, resource, properties, environ,
349 requested)
349350 yield webdav.Status(href, '200 OK', propstat=list(propstat))
350351 i += 1
2626 class SubbedProperty(webdav.Property):
2727 """Property with sub-components that can be queried."""
2828
29 def get_value_ext(self, href, resource, el, requested):
29 def get_value_ext(self, href, resource, el, environ, requested):
3030 """Get the value of a data property.
3131
3232 :param href: Resource href
3333 :param resource: Resource to get value for
3434 :param el: Element to fill in
35 :param environ: WSGI environ dict
3536 :param requested: Requested property (including subelements)
3637 """
3738 raise NotImplementedError(self.get_value_ext)
3839
3940
4041 def get_properties_with_data(data_property, href, resource, properties,
41 requested):
42 environ, requested):
4243 properties = dict(properties)
4344 properties[data_property.name] = data_property
44 return webdav.get_properties(href, resource, properties, requested)
45 return webdav.get_properties(
46 href, resource, properties, environ, requested)
4547
4648
4749 class MultiGetReporter(webdav.Reporter):
5658 def report(self, environ, body, resources_by_hrefs, properties, base_href,
5759 resource, depth):
5860 # TODO(jelmer): Verify that depth == "0"
59 # TODO(jelmer): Verify that resource is an addressbook
61 # TODO(jelmer): Verify that resource is an the right resource type
6062 requested = None
6163 hrefs = []
6264 for el in body:
7274 yield webdav.Status(href, '404 Not Found', propstat=[])
7375 else:
7476 propstat = get_properties_with_data(
75 self.data_property, href, resource, properties, requested)
77 self.data_property, href, resource, properties, environ,
78 requested)
7679 yield webdav.Status(href, '200 OK', propstat=list(propstat))
7780
7881
7982 # see https://tools.ietf.org/html/rfc4790
83
84 class UnknownCollation(Exception):
85
86 def __init__(self, collation):
87 super(UnknownCollation, self).__init__(
88 "Collation %r is not supported" % collation)
89 self.collation = collation
90
8091
8192 collations = {
8293 'i;ascii-casemap': lambda a, b: (a.decode('ascii').upper() ==
8394 b.decode('ascii').upper()),
8495 'i;octet': lambda a, b: a == b,
8596 }
97
98
99 def get_collation(name):
100 """Get a collation by name.
101
102 :param name: Collation name
103 :raises UnknownCollation: If the collation is not supported
104 """
105 try:
106 return collations[name]
107 except KeyError:
108 raise UnknownCollation(name)
2323 import logging
2424
2525 from icalendar.cal import Calendar, component_factory
26 from icalendar.prop import vText
2627 from xandikos.store import File, InvalidFileContents
28
29 # TODO(jelmer): Populate this further based on
30 # https://tools.ietf.org/html/rfc5545#3.3.11
31 _INVALID_CONTROL_CHARACTERS = ['\x0c', '\x01']
32
33
34 def validate_calendar(cal, strict=False):
35 """Validate a calendar object.
36
37 :param cal: Calendar object
38 """
39 for error in validate_component(cal, strict=strict):
40 yield error
41
42
43 def validate_component(comp, strict=False):
44 """Validate a calendar component.
45
46 :param comp: Calendar component
47 """
48 # Check text fields for invalid characters
49 for (name, value) in comp.items():
50 if isinstance(value, vText):
51 for c in _INVALID_CONTROL_CHARACTERS:
52 if c in value:
53 yield "Invalid character %s in field %s" % (
54 c.encode('unicode_escape'), name)
55 if strict:
56 for required in comp.required:
57 try:
58 comp[required]
59 except KeyError:
60 yield "Missing required field %s" % required
61 for subcomp in comp.subcomponents:
62 for error in validate_component(subcomp, strict=strict):
63 yield error
2764
2865
2966 def calendar_component_delta(old_cal, new_cal):
117154 old_component.name.upper() == "VTODO" and
118155 field.upper() == "STATUS"
119156 ):
120 yield "%s marked as %s" % (description, new_value)
157 human_readable = {
158 "NEEDS-ACTION": "needing action",
159 "COMPLETED": "complete",
160 "CANCELLED": "cancelled"}
161 yield "%s marked as %s" % (
162 description,
163 human_readable.get(new_value.upper(), new_value))
121164 elif field.upper() == 'DESCRIPTION':
122165 yield "changed description of %s" % description
123166 elif field.upper() == 'SUMMARY':
125168 elif field.upper() == 'LOCATION':
126169 yield "changed location of %s to %s" % (description, new_value)
127170 elif (old_component.name.upper() == "VTODO" and
128 field.upper() == "PERCENT-COMPLETE"):
171 field.upper() == "PERCENT-COMPLETE" and
172 new_value is not None):
129173 yield "%s marked as %d%% completed." % (
130174 description, new_value)
131175 elif field.upper() == 'DUE':
132176 yield "changed due date for %s from %s to %s" % (
133177 description, old_value.dt if old_value else 'none',
134178 new_value.dt if new_value else 'none')
179 elif field.upper() == 'DTSTART':
180 yield "changed start date/time of %s from %s to %s" % (
181 description, old_value.dt if old_value else 'none',
182 new_value.dt if new_value else 'none')
183 elif field.upper() == 'DTEND':
184 yield "changed end date/time of %s from %s to %s" % (
185 description, old_value.dt if old_value else 'none',
186 new_value.dt if new_value else 'none')
187 elif field.upper() == 'CLASS':
188 yield "changed class of %s from %s to %s" % (
189 description, old_value.lower() if old_value else 'none',
190 new_value.lower() if new_value else 'none')
135191 else:
136192 yield "modified field %s in %s" % (field, description)
137193 logging.debug("Changed %s/%s or %s/%s from %s to %s.",
150206
151207 def validate(self):
152208 """Verify that file contents are valid."""
153 self.calendar
209 cal = self.calendar
210 # TODO(jelmer): return the list of errors to the caller
211 if cal.is_broken:
212 raise InvalidFileContents(self.content_type, self.content)
213 if list(validate_calendar(cal, strict=False)):
214 raise InvalidFileContents(self.content_type, self.content)
154215
155216 @property
156217 def calendar(self):
3131 resource_type = webdav.PRINCIPAL_RESOURCE_TYPE
3232 live = False
3333
34 def get_value(self, href, resource, el):
34 def get_value(self, href, resource, el, environ):
3535 el.text = resource.get_infit_settings()
3636
3737 def set_value(self, href, resource, el):
4848 resource_type = carddav.ADDRESSBOOK_RESOURCE_TYPE
4949 in_allprops = False
5050
51 def get_value(self, href, resource, el):
51 def get_value(self, href, resource, el, environ):
5252 el.text = resource.get_addressbook_color()
5353
5454 def set_value(self, href, resource, el):
6666 in_allprops = False
6767 live = False
6868
69 def get_value(self, href, resource, el):
69 def get_value(self, href, resource, el, environ):
7070 el.text = resource.get_headervalue()
7171
7272 def set_value(self, href, resource, el):
2323 from xandikos import webdav
2424
2525
26 FEATURE = 'quota'
27
28
2629 class QuotaAvailableBytesProperty(webdav.Property):
2730 """quota-available-bytes
2831 """
3235 in_allprops = False
3336 live = True
3437
35 def get_value(self, href, resource, el):
38 def get_value(self, href, resource, el, environ):
3639 el.text = resource.get_quota_available_bytes()
3740
3841
4548 in_allprops = False
4649 live = True
4750
48 def get_value(self, href, resource, el):
51 def get_value(self, href, resource, el, environ):
4952 el.text = resource.get_quota_used_bytes()
2525
2626
2727 SCHEDULE_INBOX_RESOURCE_TYPE = '{%s}schedule-inbox' % caldav.NAMESPACE
28 SCHEDULE_OUTBOX_RESOURCE_TYPE = '{%s}schedule-outbox' % caldav.NAMESPACE
2829
2930 # Feature to advertise to indicate scheduling support.
3031 FEATURE = 'calendar-auto-schedule'
4344 CALENDAR_USER_TYPE_UNKNOWN)
4445
4546
46 class ScheduleInbox(caldav.Calendar):
47
48 resource_types = (caldav.Calendar.resource_types +
47 class ScheduleInbox(webdav.Collection):
48
49 resource_types = (webdav.Collection.resource_types +
4950 [SCHEDULE_INBOX_RESOURCE_TYPE])
50
51 def get_schedule_inbox_url(self):
52 raise NotImplementedError(self.get_schedule_inbox_url)
53
54 def get_schedule_outbox_url(self):
55 raise NotImplementedError(self.get_schedule_outbox_url)
5651
5752 def get_calendar_user_type(self):
5853 # Default, per section 2.4.2
5954 return CALENDAR_USER_TYPE_INDIVIDUAL
6055
56 def get_calendar_timezone(self):
57 """Return calendar timezone.
58
59 This should be an iCalendar object with exactly one
60 VTIMEZONE component.
61 """
62 raise NotImplementedError(self.get_calendar_timezone)
63
64 def set_calendar_timezone(self):
65 """Set calendar timezone.
66
67 This should be an iCalendar object with exactly one
68 VTIMEZONE component.
69 """
70 raise NotImplementedError(self.set_calendar_timezone)
71
72 def get_supported_calendar_components(self):
73 """Return set of supported calendar components in this calendar.
74
75 :return: iterable over component names
76 """
77 raise NotImplementedError(self.get_supported_calendar_components)
78
79 def get_supported_calendar_data_types(self):
80 """Return supported calendar data types.
81
82 :return: iterable over (content_type, version) tuples
83 """
84 raise NotImplementedError(self.get_supported_calendar_data_types)
85
86 def get_min_date_time(self):
87 """Return minimum datetime property.
88 """
89 raise NotImplementedError(self.get_min_date_time)
90
91 def get_max_date_time(self):
92 """Return maximum datetime property.
93 """
94 raise NotImplementedError(self.get_max_date_time)
95
96 def get_max_instances(self):
97 """Return maximum number of instances.
98 """
99 raise NotImplementedError(self.get_max_instances)
100
101 def get_max_attendees_per_instance(self):
102 """Return maximum number of attendees per instance.
103 """
104 raise NotImplementedError(self.get_max_attendees_per_instance)
105
106 def get_max_resource_size(self):
107 """Return max resource size."""
108 raise NotImplementedError(self.get_max_resource_size)
109
110 def get_schedule_default_calendar_url(self):
111 """Return default calendar URL.
112
113 None indicates there is no default URL.
114 """
115 return None
116
117
118 class ScheduleOutbox(webdav.Collection):
119
120 resource_types = (webdav.Collection.resource_types +
121 [SCHEDULE_OUTBOX_RESOURCE_TYPE])
122
123 def get_supported_calendar_components(self):
124 """Return set of supported calendar components in this calendar.
125
126 :return: iterable over component names
127 """
128 raise NotImplementedError(self.get_supported_calendar_components)
129
130 def get_supported_calendar_data_types(self):
131 """Return supported calendar data types.
132
133 :return: iterable over (content_type, version) tuples
134 """
135 raise NotImplementedError(self.get_supported_calendar_data_types)
136
137 def get_max_resource_size(self):
138 """Return max resource size."""
139 raise NotImplementedError(self.get_max_resource_size)
140
141 def get_min_date_time(self):
142 """Return minimum datetime property.
143 """
144 raise NotImplementedError(self.get_min_date_time)
145
146 def get_max_date_time(self):
147 """Return maximum datetime property.
148 """
149 raise NotImplementedError(self.get_max_date_time)
150
151 def get_max_attendees_per_instance(self):
152 """Return maximum number of attendees per instance.
153 """
154 raise NotImplementedError(self.get_max_attendees_per_instance)
155
61156
62157 class ScheduleInboxURLProperty(webdav.Property):
63158 """Schedule-inbox-URL property.
66161 """
67162
68163 name = '{%s}schedule-inbox-URL' % caldav.NAMESPACE
69 resource_type = caldav.CALENDAR_RESOURCE_TYPE
164 resource_type = webdav.PRINCIPAL_RESOURCE_TYPE
70165 in_allprops = True
71166
72 def get_value(self, href, resource, el):
167 def get_value(self, href, resource, el, environ):
73168 el.append(webdav.create_href(resource.get_schedule_inbox_url(), href))
74169
75170
80175 """
81176
82177 name = '{%s}schedule-outbox-URL' % caldav.NAMESPACE
83 resource_type = caldav.CALENDAR_RESOURCE_TYPE
178 resource_type = webdav.PRINCIPAL_RESOURCE_TYPE
84179 in_allprops = True
85180
86 def get_value(self, href, resource, el):
181 def get_value(self, href, resource, el, environ):
87182 el.append(webdav.create_href(resource.get_schedule_outbox_url(), href))
88183
89184
97192 resource_type = webdav.PRINCIPAL_RESOURCE_TYPE
98193 in_allprops = False
99194
100 def get_value(self, base_href, resource, el):
195 def get_value(self, base_href, resource, el, environ):
101196 for href in resource.get_calendar_user_address_set():
102197 el.append(webdav.create_href(href, base_href))
103198
108203 See https://tools.ietf.org/html/rfc6638, section 2.4.2
109204 """
110205
111 name = '{urn:ietf:params:xml:ns:caldav}calendar-user-type'
206 name = '{%s}calendar-user-type' % caldav.NAMESPACE
112207 resource_type = webdav.PRINCIPAL_RESOURCE_TYPE
113208 in_allprops = False
114209
115 def get_value(self, href, resource, el):
210 def get_value(self, href, resource, el, environ):
116211 el.text = resource.get_calendar_user_type()
212
213
214 class ScheduleDefaultCalendarURLProperty(webdav.Property):
215 """schedule-default-calendar-URL property.
216
217 See https://tools.ietf.org/html/rfc6638, section-9.2
218 """
219
220 name = '{%s}schedule-default-calendar-URL' % caldav.NAMESPACE
221 resource_types = SCHEDULE_INBOX_RESOURCE_TYPE
222 in_allprops = True
223
224 def get_value(self, href, resource, el, environ):
225 url = resource.get_schedule_default_calendar_url()
226 if url is not None:
227 el.append(webdav.create_href(url, href))
0 # Xandikos
1 # Copyright (C) 2017 Jelmer Vernooij <jelmer@jelmer.uk>, et al.
2 #
3 # This program is free software; you can redistribute it and/or
4 # modify it under the terms of the GNU General Public License
5 # as published by the Free Software Foundation; version 3
6 # of the License or (at your option) any later version of
7 # the License.
8 #
9 # This program is distributed in the hope that it will be useful,
10 # but WITHOUT ANY WARRANTY; without even the implied warranty of
11 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 # GNU General Public License for more details.
13 #
14 # You should have received a copy of the GNU General Public License
15 # along with this program; if not, write to the Free Software
16 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17 # MA 02110-1301, USA.
18
19 """Server info.
20
21 See https://www.ietf.org/archive/id/draft-douglass-server-info-03.txt
22 """
23
24 import hashlib
25
26 from xandikos import version_string
27 from xandikos import webdav
28
29 ET = webdav.ET
30
31 # Feature to advertise server-info support.
32 FEATURE = 'server-info'
33 SERVER_INFO_MIME_TYPE = 'application/server-info+xml'
34
35
36 class ServerInfo(object):
37 """Server info."""
38
39 def __init__(self):
40 self._token = None
41 self._features = []
42 self._applications = []
43
44 def add_feature(self, feature):
45 self._features.append(feature)
46 self._token = None
47
48 @property
49 def token(self):
50 if self._token is None:
51 h = hashlib.sha1sum()
52 h.update(version_string.encode('utf-8'))
53 for z in (self._features + self._applications):
54 h.update(z.encode('utf-8'))
55 self._token = h.hexdigest()
56 return self._token
57
58 def get_body(self):
59 el = ET.Element('{DAV:}server-info')
60 el.set('token', self.token)
61 server_el = ET.SubElement(el, 'server-instance-info')
62 ET.SubElement(server_el, 'name').text = 'Xandikos'
63 ET.SubElement(server_el, 'version').text = version_string
64 features_el = ET.SubElement(el, 'features')
65 for feature in self._features:
66 features_el.append(feature)
67 applications_el = ET.SubElement(el, 'applications')
68 for application in self.applications:
69 applications_el.append(application)
70 return el
0 # Xandikos
1 # Copyright (C) 2016-2017 Jelmer Vernooij <jelmer@jelmer.uk>, et al.
2 #
3 # This program is free software; you can redistribute it and/or
4 # modify it under the terms of the GNU General Public License
5 # as published by the Free Software Foundation; version 3
6 # of the License or (at your option) any later version of
7 # the License.
8 #
9 # This program is distributed in the hope that it will be useful,
10 # but WITHOUT ANY WARRANTY; without even the implied warranty of
11 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 # GNU General Public License for more details.
13 #
14 # You should have received a copy of the GNU General Public License
15 # along with this program; if not, write to the Free Software
16 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17 # MA 02110-1301, USA.
18
19 """Stores and store sets.
20
21 ETags (https://en.wikipedia.org/wiki/HTTP_ETag) used in this file
22 are always strong, and should be returned without wrapping quotes.
23 """
24
25 import mimetypes
26
27 STORE_TYPE_ADDRESSBOOK = 'addressbook'
28 STORE_TYPE_CALENDAR = 'calendar'
29 STORE_TYPE_PRINCIPAL = 'principal'
30 STORE_TYPE_SCHEDULE_INBOX = 'schedule-inbox'
31 STORE_TYPE_SCHEDULE_OUTBOX = 'schedule-outbox'
32 STORE_TYPE_OTHER = 'other'
33 VALID_STORE_TYPES = (
34 STORE_TYPE_ADDRESSBOOK,
35 STORE_TYPE_CALENDAR,
36 STORE_TYPE_PRINCIPAL,
37 STORE_TYPE_SCHEDULE_INBOX,
38 STORE_TYPE_SCHEDULE_OUTBOX,
39 STORE_TYPE_OTHER)
40
41 MIMETYPES = mimetypes.MimeTypes()
42 MIMETYPES.add_type('text/calendar', '.ics')
43 MIMETYPES.add_type('text/vcard', '.vcf')
44
45 DEFAULT_MIME_TYPE = 'application/octet-stream'
46
47
48 class File(object):
49 """A file type handler."""
50
51 def __init__(self, content, content_type):
52 self.content = content
53 self.content_type = content_type
54
55 def validate(self):
56 """Verify that file contents are valid.
57
58 :raise InvalidFileContents: Raised if a file is not valid
59 """
60 pass
61
62 def describe(self, name):
63 """Describe the contents of this file.
64
65 Used in e.g. commit messages.
66 """
67 return name
68
69 def get_uid(self):
70 """Return UID.
71
72 :raise NotImplementedError: If UIDs aren't supported for this format
73 :raise KeyError: If there is no UID set on this file
74 :raise InvalidFileContents: If the file is misformatted
75 :return: UID
76 """
77 raise NotImplementedError(self.get_uid)
78
79 def describe_delta(self, name, previous):
80 """Describe the important difference between this and previous one.
81
82 :param name: File name
83 :param previous: Previous file to compare to.
84 :raise InvalidFileContents: If the file is misformatted
85 :return: List of strings describing change
86 """
87 assert name is not None
88 item_description = self.describe(name)
89 assert item_description is not None
90 if previous is None:
91 yield "Added " + item_description
92 else:
93 yield "Modified " + item_description
94
95
96 def open_by_content_type(content, content_type, extra_file_handlers):
97 """Open a file based on content type.
98
99 :param content: list of bytestrings with content
100 :param content_type: MIME type
101 :return: File instance
102 """
103 return extra_file_handlers.get(content_type.split(';')[0], File)(
104 content, content_type)
105
106
107 def open_by_extension(content, name, extra_file_handlers):
108 """Open a file based on the filename extension.
109
110 :param content: list of bytestrings with content
111 :param name: Name of file to open
112 :return: File instance
113 """
114 (mime_type, _) = MIMETYPES.guess_type(name)
115 if mime_type is None:
116 mime_type = DEFAULT_MIME_TYPE
117 return open_by_content_type(content, mime_type,
118 extra_file_handlers=extra_file_handlers)
119
120
121 class DuplicateUidError(Exception):
122 """UID already exists in store."""
123
124 def __init__(self, uid, existing_name, new_name):
125 self.uid = uid
126 self.existing_name = existing_name
127 self.new_name = new_name
128
129
130 class NoSuchItem(Exception):
131 """No such item."""
132
133 def __init__(self, name):
134 self.name = name
135
136
137 class InvalidETag(Exception):
138 """Unexpected value for etag."""
139
140 def __init__(self, name, expected_etag, got_etag):
141 self.name = name
142 self.expected_etag = expected_etag
143 self.got_etag = got_etag
144
145
146 class NotStoreError(Exception):
147 """Not a store."""
148
149 def __init__(self, path):
150 self.path = path
151
152
153 class InvalidFileContents(Exception):
154 """Invalid file contents."""
155
156 def __init__(self, content_type, data):
157 self.content_type = content_type
158 self.data = data
159
160
161 class Store(object):
162 """A object store."""
163
164 def __init__(self):
165 self.extra_file_handlers = {}
166
167 def load_extra_file_handler(self, file_handler):
168 self.extra_file_handlers[file_handler.content_type] = file_handler
169
170 def iter_with_etag(self):
171 """Iterate over all items in the store with etag.
172
173 :yield: (name, content_type, etag) tuples
174 """
175 raise NotImplementedError(self.iter_with_etag)
176
177 def get_file(self, name, content_type=None, etag=None):
178 """Get the contents of an object.
179
180 :return: A File object
181 """
182 if content_type is None:
183 return open_by_extension(
184 self._get_raw(name, etag), name,
185 extra_file_handlers=self.extra_file_handlers)
186 else:
187 return open_by_content_type(
188 self._get_raw(name, etag), content_type,
189 extra_file_handlers=self.extra_file_handlers)
190
191 def _get_raw(self, name, etag):
192 """Get the raw contents of an object.
193
194 :return: raw contents
195 """
196 raise NotImplementedError(self._get_raw)
197
198 def get_ctag(self):
199 """Return the ctag for this store."""
200 raise NotImplementedError(self.get_ctag)
201
202 def import_one(self, name, data, message=None, author=None,
203 replace_etag=None):
204 """Import a single object.
205
206 :param name: Name of the object
207 :param data: serialized object as list of bytes
208 :param message: Commit message
209 :param author: Optional author
210 :param replace_etag: Etag to replace
211 :raise NameExists: when the name already exists
212 :raise DuplicateUidError: when the uid already exists
213 :return: (name, etag)
214 """
215 raise NotImplementedError(self.import_one)
216
217 def delete_one(self, name, message=None, author=None, etag=None):
218 """Delete an item.
219
220 :param name: Filename to delete
221 :param author: Optional author
222 :param message: Commit message
223 :param etag: Optional mandatory etag of object to remove
224 :raise NoSuchItem: when the item doesn't exist
225 :raise InvalidETag: If the specified ETag doesn't match the current
226 """
227 raise NotImplementedError(self.delete_one)
228
229 def set_type(self, store_type):
230 """Set store type.
231
232 :param store_type: New store type (one of VALID_STORE_TYPES)
233 """
234 raise NotImplementedError(self.set_type)
235
236 def get_type(self):
237 """Get type of this store.
238
239 :return: one of VALID_STORE_TYPES
240 """
241 ret = STORE_TYPE_OTHER
242 for (name, content_type, etag) in self.iter_with_etag():
243 if content_type == 'text/calendar':
244 ret = STORE_TYPE_CALENDAR
245 elif content_type == 'text/vcard':
246 ret = STORE_TYPE_ADDRESSBOOK
247 return ret
248
249 def set_description(self, description):
250 """Set the extended description of this store.
251
252 :param description: String with description
253 """
254 raise NotImplementedError(self.set_description)
255
256 def get_description(self):
257 """Get the extended description of this store.
258 """
259 raise NotImplementedError(self.get_description)
260
261 def get_displayname(self):
262 """Get the display name of this store.
263 """
264 raise NotImplementedError(self.get_displayname)
265
266 def set_displayname(self):
267 """Set the display name of this store.
268 """
269 raise NotImplementedError(self.set_displayname)
270
271 def get_color(self):
272 """Get the color code for this store."""
273 raise NotImplementedError(self.get_color)
274
275 def set_color(self, color):
276 """Set the color code for this store."""
277 raise NotImplementedError(self.set_color)
278
279 def iter_changes(self, old_ctag, new_ctag):
280 """Get changes between two versions of this store.
281
282 :param old_ctag: Old ctag (None for empty Store)
283 :param new_ctag: New ctag
284 :return: Iterator over (name, content_type, old_etag, new_etag)
285 """
286 raise NotImplementedError(self.iter_changes)
287
288 def get_comment(self):
289 """Retrieve store comment.
290
291 :return: Comment
292 """
293 raise NotImplementedError(self.get_comment)
294
295 def set_comment(self, comment):
296 """Set comment.
297
298 :param comment: New comment to set
299 """
300 raise NotImplementedError(self.set_comment)
301
302 def destroy(self):
303 """Destroy this store."""
304 raise NotImplementedError(self.destroy)
305
306 def subdirectories(self):
307 """Returns subdirectories to probe for other stores.
308
309 :return: List of names
310 """
311 raise NotImplementedError(self.subdirectories)
312
313
314 def open_store(location):
315 """Open store from a location string.
316
317 :param location: Location string to open
318 :return: A `Store`
319 """
320 # For now, just support opening git stores
321 from .git import GitStore
322 return GitStore.open_from_path(location)
0 # Xandikos
1 # Copyright (C) 2016-2017 Jelmer Vernooij <jelmer@jelmer.uk>, et al.
2 #
3 # This program is free software; you can redistribute it and/or
4 # modify it under the terms of the GNU General Public License
5 # as published by the Free Software Foundation; version 3
6 # of the License or (at your option) any later version of
7 # the License.
8 #
9 # This program is distributed in the hope that it will be useful,
10 # but WITHOUT ANY WARRANTY; without even the implied warranty of
11 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 # GNU General Public License for more details.
13 #
14 # You should have received a copy of the GNU General Public License
15 # along with this program; if not, write to the Free Software
16 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17 # MA 02110-1301, USA.
18
19 """Collection configuration file.
20 """
21
22 import configparser
23
24 FILENAME = '.xandikos'
25
26
27 class CollectionConfig(object):
28
29 def __init__(self, cp=None):
30 if cp is None:
31 cp = configparser.ConfigParser()
32 self._configparser = cp
33
34 @classmethod
35 def from_file(cls, f):
36 cp = configparser.ConfigParser()
37 cp.read_file(f)
38 return CollectionConfig(cp)
39
40 def get_color(self):
41 return self._configparser['DEFAULT']['color']
42
43 def get_comment(self):
44 return self._configparser['DEFAULT']['comment']
45
46 def get_displayname(self):
47 return self._configparser['DEFAULT']['displayname']
48
49 def get_description(self):
50 return self._configparser['DEFAULT']['description']
0 # Xandikos
1 # Copyright (C) 2016-2017 Jelmer Vernooij <jelmer@jelmer.uk>, et al.
2 #
3 # This program is free software; you can redistribute it and/or
4 # modify it under the terms of the GNU General Public License
5 # as published by the Free Software Foundation; version 3
6 # of the License or (at your option) any later version of
7 # the License.
8 #
9 # This program is distributed in the hope that it will be useful,
10 # but WITHOUT ANY WARRANTY; without even the implied warranty of
11 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 # GNU General Public License for more details.
13 #
14 # You should have received a copy of the GNU General Public License
15 # along with this program; if not, write to the Free Software
16 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17 # MA 02110-1301, USA.
18
19 """Git store.
20 """
21
22 import logging
23 import os
24 import shutil
25 import stat
26 import uuid
27
28 from . import (
29 DEFAULT_MIME_TYPE,
30 MIMETYPES,
31 Store,
32 DuplicateUidError,
33 InvalidETag,
34 InvalidFileContents,
35 NoSuchItem,
36 NotStoreError,
37 VALID_STORE_TYPES,
38 open_by_content_type,
39 open_by_extension,
40 )
41 from .config import CollectionConfig
42
43
44 from dulwich.file import GitFile
45 from dulwich.index import (
46 Index,
47 IndexEntry,
48 index_entry_from_stat,
49 write_index_dict,
50 )
51 from dulwich.objects import Blob, Tree
52 from dulwich.pack import SHA1Writer
53 import dulwich.repo
54
55 _DEFAULT_COMMITTER_IDENTITY = b'Xandikos <xandikos>'
56 DEFAULT_ENCODING = 'utf-8'
57
58
59 logger = logging.getLogger(__name__)
60
61
62 class locked_index(object):
63
64 def __init__(self, path):
65 self._path = path
66
67 def __enter__(self):
68 self._file = GitFile(self._path, 'wb')
69 self._index = Index(self._path)
70 return self._index
71
72 def __exit__(self, exc_type, exc_value, traceback):
73 f = SHA1Writer(self._file)
74 write_index_dict(f, self._index._byname)
75 f.close()
76
77
78 class GitStore(Store):
79 """A Store backed by a Git Repository.
80 """
81
82 def __init__(self, repo, ref=b'refs/heads/master',
83 check_for_duplicate_uids=True):
84 super(GitStore, self).__init__()
85 self.ref = ref
86 self.repo = repo
87 # Maps uids to (sha, fname)
88 self._uid_to_fname = {}
89 self._check_for_duplicate_uids = check_for_duplicate_uids
90 # Set of blob ids that have already been scanned
91 self._fname_to_uid = {}
92
93 @property
94 def config(self):
95 return CollectionConfig()
96
97 def __repr__(self):
98 return "%s(%r, ref=%r)" % (type(self).__name__, self.repo, self.ref)
99
100 @property
101 def path(self):
102 return self.repo.path
103
104 def _check_duplicate(self, uid, name, replace_etag):
105 if uid is not None and self._check_for_duplicate_uids:
106 self._scan_uids()
107 try:
108 (existing_name, _) = self._uid_to_fname[uid]
109 except KeyError:
110 pass
111 else:
112 if existing_name != name:
113 raise DuplicateUidError(uid, existing_name, name)
114
115 try:
116 etag = self._get_etag(name)
117 except KeyError:
118 etag = None
119 if replace_etag is not None and etag != replace_etag:
120 raise InvalidETag(name, etag, replace_etag)
121 return etag
122
123 def import_one(self, name, content_type, data, message=None, author=None,
124 replace_etag=None):
125 """Import a single object.
126
127 :param name: name of the object
128 :param content_type: Content type
129 :param data: serialized object as list of bytes
130 :param message: Commit message
131 :param author: Optional author
132 :param replace_etag: optional etag of object to replace
133 :raise InvalidETag: when the name already exists but with different
134 etag
135 :raise DuplicateUidError: when the uid already exists
136 :return: etag
137 """
138 if content_type is None:
139 fi = open_by_extension(data, name, self.extra_file_handlers)
140 else:
141 fi = open_by_content_type(
142 data, content_type, self.extra_file_handlers)
143 if name is None:
144 name = str(uuid.uuid4())
145 extension = MIMETYPES.guess_extension(content_type)
146 if extension is not None:
147 name += extension
148 fi.validate()
149 try:
150 uid = fi.get_uid()
151 except (KeyError, NotImplementedError):
152 uid = None
153 self._check_duplicate(uid, name, replace_etag)
154 if message is None:
155 try:
156 old_fi = self.get_file(name, content_type, replace_etag)
157 except KeyError:
158 old_fi = None
159 message = '\n'.join(fi.describe_delta(name, old_fi))
160 etag = self._import_one(name, data, message, author=author)
161 return (name, etag.decode('ascii'))
162
163 def _get_raw(self, name, etag=None):
164 """Get the raw contents of an object.
165
166 :param name: Name of the item
167 :param etag: Optional etag
168 :return: raw contents as chunks
169 """
170 if etag is None:
171 etag = self._get_etag(name)
172 blob = self.repo.object_store[etag.encode('ascii')]
173 return blob.chunked
174
175 def _scan_uids(self):
176 removed = set(self._fname_to_uid.keys())
177 for (name, mode, sha) in self._iterblobs():
178 etag = sha.decode('ascii')
179 if name in removed:
180 removed.remove(name)
181 if (name in self._fname_to_uid and
182 self._fname_to_uid[name][0] == etag):
183 continue
184 blob = self.repo.object_store[sha]
185 fi = open_by_extension(blob.chunked, name,
186 self.extra_file_handlers)
187 try:
188 uid = fi.get_uid()
189 except KeyError:
190 logger.warning('No UID found in file %s', name)
191 uid = None
192 except InvalidFileContents:
193 logging.warning('Unable to parse file %s', name)
194 uid = None
195 except NotImplementedError:
196 # This file type doesn't support UIDs
197 uid = None
198 self._fname_to_uid[name] = (etag, uid)
199 if uid is not None:
200 self._uid_to_fname[uid] = (name, etag)
201 for name in removed:
202 (unused_etag, uid) = self._fname_to_uid[name]
203 if uid is not None:
204 del self._uid_to_fname[uid]
205 del self._fname_to_uid[name]
206
207 def _iterblobs(self, ctag=None):
208 raise NotImplementedError(self._iterblobs)
209
210 def iter_with_etag(self, ctag=None):
211 """Iterate over all items in the store with etag.
212
213 :param ctag: Ctag to iterate for
214 :yield: (name, content_type, etag) tuples
215 """
216 for (name, mode, sha) in self._iterblobs(ctag):
217 (mime_type, _) = MIMETYPES.guess_type(name)
218 if mime_type is None:
219 mime_type = DEFAULT_MIME_TYPE
220 yield (name, mime_type, sha.decode('ascii'))
221
222 @classmethod
223 def create(cls, path):
224 """Create a new store backed by a Git repository on disk.
225
226 :return: A `GitStore`
227 """
228 raise NotImplementedError(cls.create)
229
230 @classmethod
231 def open_from_path(cls, path):
232 """Open a GitStore from a path.
233
234 :param path: Path
235 :return: A `GitStore`
236 """
237 try:
238 return cls.open(dulwich.repo.Repo(path))
239 except dulwich.repo.NotGitRepository:
240 raise NotStoreError(path)
241
242 @classmethod
243 def open(cls, repo):
244 """Open a GitStore given a Repo object.
245
246 :param repo: A Dulwich `Repo`
247 :return: A `GitStore`
248 """
249 if repo.has_index():
250 return TreeGitStore(repo)
251 else:
252 return BareGitStore(repo)
253
254 def get_description(self):
255 """Get extended description.
256
257 :return: repository description as string
258 """
259 try:
260 return self.config.get_description()
261 except KeyError:
262 desc = self.repo.get_description()
263 if desc is not None:
264 desc = desc.decode(DEFAULT_ENCODING)
265 return desc
266
267 def set_description(self, description):
268 """Set extended description.
269
270 :param description: repository description as string
271 """
272 return self.repo.set_description(description.encode(DEFAULT_ENCODING))
273
274 def set_comment(self, comment):
275 """Set comment.
276
277 :param comment: Comment
278 """
279 config = self.repo.get_config()
280 config.set(b'xandikos', b'comment', comment.encode(DEFAULT_ENCODING))
281 config.write_to_path()
282
283 def get_comment(self):
284 """Get comment.
285
286 :return: Comment
287 """
288 try:
289 return self.config.get_comment()
290 except KeyError:
291 config = self.repo.get_config()
292 try:
293 comment = config.get(b'xandikos', b'comment')
294 except KeyError:
295 return None
296 else:
297 return comment.decode(DEFAULT_ENCODING)
298
299 def get_color(self):
300 """Get color.
301
302 :return: A Color code, or None
303 """
304 try:
305 return self.config.get_color()
306 except KeyError:
307 config = self.repo.get_config()
308 try:
309 color = config.get(b'xandikos', b'color')
310 except KeyError:
311 return None
312 else:
313 return color.decode(DEFAULT_ENCODING)
314
315 def set_color(self, color):
316 """Set the color code for this store."""
317 config = self.repo.get_config()
318 # Strip leading # to work around
319 # https://github.com/jelmer/dulwich/issues/511
320 # TODO(jelmer): Drop when that bug gets fixed.
321 config.set(
322 b'xandikos', b'color',
323 color.lstrip('#').encode(DEFAULT_ENCODING) if color else b'')
324 config.write_to_path()
325
326 def get_displayname(self):
327 """Get display name.
328
329 :return: The display name, or None if not set
330 """
331 try:
332 return self.config.get_displayname()
333 except KeyError:
334 config = self.repo.get_config()
335 try:
336 displayname = config.get(b'xandikos', b'displayname')
337 except KeyError:
338 return None
339 else:
340 return displayname.decode(DEFAULT_ENCODING)
341
342 def set_displayname(self, displayname):
343 """Set the display name.
344
345 :param displayname: New display name
346 """
347 config = self.repo.get_config()
348 config.set(b'xandikos', b'displayname',
349 displayname.encode(DEFAULT_ENCODING))
350 config.write_to_path()
351
352 def set_type(self, store_type):
353 """Set store type.
354
355 :param store_type: New store type (one of VALID_STORE_TYPES)
356 """
357 config = self.repo.get_config()
358 config.set(b'xandikos', b'type', store_type.encode(DEFAULT_ENCODING))
359 config.write_to_path()
360
361 def get_type(self):
362 """Get store type.
363
364 This looks in git config first, then falls back to guessing.
365 """
366 config = self.repo.get_config()
367 try:
368 store_type = config.get(b'xandikos', b'type')
369 except KeyError:
370 return super(GitStore, self).get_type()
371 else:
372 store_type = store_type.decode(DEFAULT_ENCODING)
373 if store_type not in VALID_STORE_TYPES:
374 logging.warning(
375 'Invalid store type %s set for %r.',
376 store_type, self.repo)
377 return store_type
378
379 def iter_changes(self, old_ctag, new_ctag):
380 """Get changes between two versions of this store.
381
382 :param old_ctag: Old ctag (None for empty Store)
383 :param new_ctag: New ctag
384 :return: Iterator over (name, content_type, old_etag, new_etag)
385 """
386 if old_ctag is None:
387 t = Tree()
388 self.repo.object_store.add_object(t)
389 old_ctag = t.id.decode('ascii')
390 previous = {
391 name: (content_type, etag)
392 for (name, content_type, etag) in self.iter_with_etag(old_ctag)
393 }
394 for (name, new_content_type, new_etag) in (
395 self.iter_with_etag(new_ctag)):
396 try:
397 (old_content_type, old_etag) = previous[name]
398 except KeyError:
399 old_etag = None
400 else:
401 assert old_content_type == new_content_type
402 if old_etag != new_etag:
403 yield (name, new_content_type, old_etag, new_etag)
404 if old_etag is not None:
405 del previous[name]
406 for (name, (old_content_type, old_etag)) in previous.items():
407 yield (name, old_content_type, old_etag, None)
408
409 def destroy(self):
410 """Destroy this store."""
411 shutil.rmtree(self.path)
412
413
414 class BareGitStore(GitStore):
415 """A Store backed by a bare git repository."""
416
417 def _get_current_tree(self):
418 try:
419 ref_object = self.repo[self.ref]
420 except KeyError:
421 return Tree()
422 if isinstance(ref_object, Tree):
423 return ref_object
424 else:
425 return self.repo.object_store[ref_object.tree]
426
427 def _get_etag(self, name):
428 tree = self._get_current_tree()
429 name = name.encode(DEFAULT_ENCODING)
430 return tree[name][1].decode('ascii')
431
432 def get_ctag(self):
433 """Return the ctag for this store."""
434 return self._get_current_tree().id.decode('ascii')
435
436 def _iterblobs(self, ctag=None):
437 if ctag is None:
438 tree = self._get_current_tree()
439 else:
440 tree = self.repo.object_store[ctag.encode('ascii')]
441 for (name, mode, sha) in tree.iteritems():
442 name = name.decode(DEFAULT_ENCODING)
443 yield (name, mode, sha)
444
445 @classmethod
446 def create_memory(cls):
447 """Create a new store backed by a memory repository.
448
449 :return: A `GitStore`
450 """
451 return cls(dulwich.repo.MemoryRepo())
452
453 def _commit_tree(self, tree_id, message, author=None):
454 try:
455 committer = self.repo._get_user_identity()
456 except KeyError:
457 committer = _DEFAULT_COMMITTER_IDENTITY
458 return self.repo.do_commit(message=message, tree=tree_id, ref=self.ref,
459 committer=committer, author=author)
460
461 def _import_one(self, name, data, message, author=None):
462 """Import a single object.
463
464 :param name: Optional name of the object
465 :param data: serialized object as bytes
466 :param message: optional commit message
467 :param author: optional author
468 :return: etag
469 """
470 b = Blob()
471 b.chunked = data
472 tree = self._get_current_tree()
473 name_enc = name.encode(DEFAULT_ENCODING)
474 tree[name_enc] = (0o644 | stat.S_IFREG, b.id)
475 self.repo.object_store.add_objects([(tree, ''), (b, name_enc)])
476 self._commit_tree(tree.id, message.encode(DEFAULT_ENCODING),
477 author=author)
478 return b.id
479
480 def delete_one(self, name, message=None, author=None, etag=None):
481 """Delete an item.
482
483 :param name: Filename to delete
484 :param message; Commit message
485 :param author: Optional author to store
486 :param etag: Optional mandatory etag of object to remove
487 :raise NoSuchItem: when the item doesn't exist
488 :raise InvalidETag: If the specified ETag doesn't match the curren
489 """
490 tree = self._get_current_tree()
491 name_enc = name.encode(DEFAULT_ENCODING)
492 try:
493 current_sha = tree[name_enc][1]
494 except KeyError:
495 raise NoSuchItem(name)
496 if etag is not None and current_sha != etag.encode('ascii'):
497 raise InvalidETag(name, etag, current_sha.decode('ascii'))
498 del tree[name_enc]
499 self.repo.object_store.add_objects([(tree, '')])
500 if message is None:
501 fi = open_by_extension(
502 self.repo.object_store[current_sha].chunked, name,
503 self.extra_file_handlers)
504 message = "Delete " + fi.describe(name)
505 self._commit_tree(tree.id, message.encode(DEFAULT_ENCODING),
506 author=author)
507
508 @classmethod
509 def create(cls, path):
510 """Create a new store backed by a Git repository on disk.
511
512 :return: A `GitStore`
513 """
514 os.mkdir(path)
515 return cls(dulwich.repo.Repo.init_bare(path))
516
517 def subdirectories(self):
518 """Returns subdirectories to probe for other stores.
519
520 :return: List of names
521 """
522 # Or perhaps just return all subdirectories but filter out
523 # Git-owned ones?
524 return []
525
526
527 class TreeGitStore(GitStore):
528 """A Store that backs onto a treefull Git repository."""
529
530 @classmethod
531 def create(cls, path, bare=True):
532 """Create a new store backed by a Git repository on disk.
533
534 :return: A `GitStore`
535 """
536 os.mkdir(path)
537 return cls(dulwich.repo.Repo.init(path))
538
539 def _get_etag(self, name):
540 index = self.repo.open_index()
541 name = name.encode(DEFAULT_ENCODING)
542 return index[name].sha.decode('ascii')
543
544 def _commit_tree(self, index, message, author=None):
545 tree = index.commit(self.repo.object_store)
546 try:
547 committer = self.repo._get_user_identity()
548 except KeyError:
549 committer = _DEFAULT_COMMITTER_IDENTITY
550 return self.repo.do_commit(message=message, committer=committer,
551 author=author, tree=tree)
552
553 def _import_one(self, name, data, message, author=None):
554 """Import a single object.
555
556 :param name: name of the object
557 :param data: serialized object as list of bytes
558 :param message: Commit message
559 :param author: Optional author
560 :return: etag
561 """
562 with locked_index(self.repo.index_path()) as index:
563 p = os.path.join(self.repo.path, name)
564 with open(p, 'wb') as f:
565 f.writelines(data)
566 st = os.lstat(p)
567 blob = Blob.from_string(b''.join(data))
568 self.repo.object_store.add_object(blob)
569 index[name.encode(DEFAULT_ENCODING)] = IndexEntry(
570 *index_entry_from_stat(st, blob.id, 0))
571 self._commit_tree(
572 index, message.encode(DEFAULT_ENCODING),
573 author=author)
574 return blob.id
575
576 def delete_one(self, name, message=None, author=None, etag=None):
577 """Delete an item.
578
579 :param name: Filename to delete
580 :param message: Commit message
581 :param author: Optional author
582 :param etag: Optional mandatory etag of object to remove
583 :raise NoSuchItem: when the item doesn't exist
584 :raise InvalidETag: If the specified ETag doesn't match the curren
585 """
586 p = os.path.join(self.repo.path, name)
587 try:
588 with open(p, 'rb') as f:
589 current_blob = Blob.from_string(f.read())
590 except IOError:
591 raise NoSuchItem(name)
592 if message is None:
593 fi = open_by_extension(current_blob.chunked, name,
594 self.extra_file_handlers)
595 message = 'Delete ' + fi.describe(name)
596 if etag is not None:
597 with open(p, 'rb') as f:
598 current_etag = current_blob.id
599 if etag.encode('ascii') != current_etag:
600 raise InvalidETag(name, etag, current_etag.decode('ascii'))
601 with locked_index(self.repo.index_path()) as index:
602 os.unlink(p)
603 del index[name.encode(DEFAULT_ENCODING)]
604 self._commit_tree(index, message.encode(DEFAULT_ENCODING),
605 author=author)
606
607 def get_ctag(self):
608 """Return the ctag for this store."""
609 index = self.repo.open_index()
610 return index.commit(self.repo.object_store).decode('ascii')
611
612 def _iterblobs(self, ctag=None):
613 """Iterate over all items in the store with etag.
614
615 :yield: (name, etag) tuples
616 """
617 if ctag is not None:
618 tree = self.repo.object_store[ctag.encode('ascii')]
619 for (name, mode, sha) in tree.iteritems():
620 name = name.decode(DEFAULT_ENCODING)
621 yield (name, mode, sha)
622 else:
623 index = self.repo.open_index()
624 for (name, sha, mode) in index.iterblobs():
625 name = name.decode(DEFAULT_ENCODING)
626 yield (name, mode, sha)
627
628 def subdirectories(self):
629 """Returns subdirectories to probe for other stores.
630
631 :return: List of names
632 """
633 ret = []
634 for name in os.listdir(self.path):
635 if name == dulwich.repo.CONTROLDIR:
636 continue
637 p = os.path.join(self.path, name)
638 if os.path.isdir(p):
639 ret.append(name)
640 return ret
+0
-868
xandikos/store.py less more
0 # Xandikos
1 # Copyright (C) 2016-2017 Jelmer Vernooij <jelmer@jelmer.uk>, et al.
2 #
3 # This program is free software; you can redistribute it and/or
4 # modify it under the terms of the GNU General Public License
5 # as published by the Free Software Foundation; version 3
6 # of the License or (at your option) any later version of
7 # the License.
8 #
9 # This program is distributed in the hope that it will be useful,
10 # but WITHOUT ANY WARRANTY; without even the implied warranty of
11 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 # GNU General Public License for more details.
13 #
14 # You should have received a copy of the GNU General Public License
15 # along with this program; if not, write to the Free Software
16 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
17 # MA 02110-1301, USA.
18
19 """Stores and store sets.
20
21 ETags (https://en.wikipedia.org/wiki/HTTP_ETag) used in this file
22 are always strong, and should be returned without wrapping quotes.
23 """
24
25 import logging
26 import mimetypes
27 import os
28 import shutil
29 import stat
30 import uuid
31
32 from dulwich.objects import Blob, Tree
33 import dulwich.repo
34
35 _DEFAULT_COMMITTER_IDENTITY = b'Xandikos <xandikos>'
36
37 STORE_TYPE_ADDRESSBOOK = 'addressbook'
38 STORE_TYPE_CALENDAR = 'calendar'
39 STORE_TYPE_PRINCIPAL = 'principal'
40 STORE_TYPE_OTHER = 'other'
41 VALID_STORE_TYPES = (
42 STORE_TYPE_ADDRESSBOOK,
43 STORE_TYPE_CALENDAR,
44 STORE_TYPE_PRINCIPAL,
45 STORE_TYPE_OTHER)
46
47 MIMETYPES = mimetypes.MimeTypes()
48 MIMETYPES.add_type('text/calendar', '.ics')
49 MIMETYPES.add_type('text/vcard', '.vcf')
50
51 DEFAULT_MIME_TYPE = 'application/octet-stream'
52 DEFAULT_ENCODING = 'utf-8'
53
54
55 logger = logging.getLogger(__name__)
56
57
58 class File(object):
59 """A file type handler."""
60
61 def __init__(self, content, content_type):
62 self.content = content
63 self.content_type = content_type
64
65 def validate(self):
66 """Verify that file contents are valid.
67
68 :raise InvalidFileContents: Raised if a file is not valid
69 """
70 pass
71
72 def describe(self, name):
73 """Describe the contents of this file.
74
75 Used in e.g. commit messages.
76 """
77 return name
78
79 def get_uid(self):
80 """Return UID.
81
82 :raise NotImplementedError: If UIDs aren't supported for this format
83 :raise KeyError: If there is no UID set on this file
84 :raise InvalidFileContents: If the file is misformatted
85 :return: UID
86 """
87 raise NotImplementedError(self.get_uid)
88
89 def describe_delta(self, name, previous):
90 """Describe the important difference between this and previous one.
91
92 :param name: File name
93 :param previous: Previous file to compare to.
94 :raise InvalidFileContents: If the file is misformatted
95 :return: List of strings describing change
96 """
97 assert name is not None
98 item_description = self.describe(name)
99 assert item_description is not None
100 if previous is None:
101 yield "Added " + item_description
102 else:
103 yield "Modified " + item_description
104
105
106 def open_by_content_type(content, content_type, extra_file_handlers):
107 """Open a file based on content type.
108
109 :param content: list of bytestrings with content
110 :param content_type: MIME type
111 :return: File instance
112 """
113 return extra_file_handlers.get(content_type.split(';')[0], File)(
114 content, content_type)
115
116
117 def open_by_extension(content, name, extra_file_handlers):
118 """Open a file based on the filename extension.
119
120 :param content: list of bytestrings with content
121 :param name: Name of file to open
122 :return: File instance
123 """
124 (mime_type, _) = MIMETYPES.guess_type(name)
125 if mime_type is None:
126 mime_type = DEFAULT_MIME_TYPE
127 return open_by_content_type(content, mime_type,
128 extra_file_handlers=extra_file_handlers)
129
130
131 class DuplicateUidError(Exception):
132 """UID already exists in store."""
133
134 def __init__(self, uid, existing_name, new_name):
135 self.uid = uid
136 self.existing_name = existing_name
137 self.new_name = new_name
138
139
140 class NoSuchItem(Exception):
141 """No such item."""
142
143 def __init__(self, name):
144 self.name = name
145
146
147 class InvalidETag(Exception):
148 """Unexpected value for etag."""
149
150 def __init__(self, name, expected_etag, got_etag):
151 self.name = name
152 self.expected_etag = expected_etag
153 self.got_etag = got_etag
154
155
156 class NotStoreError(Exception):
157 """Not a store."""
158
159 def __init__(self, path):
160 self.path = path
161
162
163 class InvalidFileContents(Exception):
164 """Invalid file contents."""
165
166 def __init__(self, content_type, data):
167 self.content_type = content_type
168 self.data = data
169
170
171 class Store(object):
172 """A object store."""
173
174 def __init__(self):
175 self.extra_file_handlers = {}
176
177 def load_extra_file_handler(self, file_handler):
178 self.extra_file_handlers[file_handler.content_type] = file_handler
179
180 def iter_with_etag(self):
181 """Iterate over all items in the store with etag.
182
183 :yield: (name, content_type, etag) tuples
184 """
185 raise NotImplementedError(self.iter_with_etag)
186
187 def get_file(self, name, content_type=None, etag=None):
188 """Get the contents of an object.
189
190 :return: A File object
191 """
192 if content_type is None:
193 return open_by_extension(
194 self._get_raw(name, etag), name,
195 extra_file_handlers=self.extra_file_handlers)
196 else:
197 return open_by_content_type(
198 self._get_raw(name, etag), content_type,
199 extra_file_handlers=self.extra_file_handlers)
200
201 def _get_raw(self, name, etag):
202 """Get the raw contents of an object.
203
204 :return: raw contents
205 """
206 raise NotImplementedError(self._get_raw)
207
208 def get_ctag(self):
209 """Return the ctag for this store."""
210 raise NotImplementedError(self.get_ctag)
211
212 def import_one(self, name, data, message=None, author=None,
213 replace_etag=None):
214 """Import a single object.
215
216 :param name: Name of the object
217 :param data: serialized object as list of bytes
218 :param message: Commit message
219 :param author: Optional author
220 :param replace_etag: Etag to replace
221 :raise NameExists: when the name already exists
222 :raise DuplicateUidError: when the uid already exists
223 :return: (name, etag)
224 """
225 raise NotImplementedError(self.import_one)
226
227 def delete_one(self, name, message=None, author=None, etag=None):
228 """Delete an item.
229
230 :param name: Filename to delete
231 :param author: Optional author
232 :param message: Commit message
233 :param etag: Optional mandatory etag of object to remove
234 :raise NoSuchItem: when the item doesn't exist
235 :raise InvalidETag: If the specified ETag doesn't match the current
236 """
237 raise NotImplementedError(self.delete_one)
238
239 def set_type(self, store_type):
240 """Set store type.
241
242 :param store_type: New store type (one of VALID_STORE_TYPES)
243 """
244 raise NotImplementedError(self.set_type)
245
246 def get_type(self):
247 """Get type of this store.
248
249 :return: one of VALID_STORE_TYPES
250 """
251 ret = STORE_TYPE_OTHER
252 for (name, content_type, etag) in self.iter_with_etag():
253 if content_type == 'text/calendar':
254 ret = STORE_TYPE_CALENDAR
255 elif content_type == 'text/vcard':
256 ret = STORE_TYPE_ADDRESSBOOK
257 return ret
258
259 def set_description(self, description):
260 """Set the extended description of this store.
261
262 :param description: String with description
263 """
264 raise NotImplementedError(self.set_description)
265
266 def get_description(self):
267 """Get the extended description of this store.
268 """
269 raise NotImplementedError(self.get_description)
270
271 def get_displayname(self):
272 """Get the display name of this store.
273 """
274 raise NotImplementedError(self.get_displayname)
275
276 def set_displayname(self):
277 """Set the display name of this store.
278 """
279 raise NotImplementedError(self.set_displayname)
280
281 def get_color(self):
282 """Get the color code for this store."""
283 raise NotImplementedError(self.get_color)
284
285 def set_color(self, color):
286 """Set the color code for this store."""
287 raise NotImplementedError(self.set_color)
288
289 def iter_changes(self, old_ctag, new_ctag):
290 """Get changes between two versions of this store.
291
292 :param old_ctag: Old ctag (None for empty Store)
293 :param new_ctag: New ctag
294 :return: Iterator over (name, content_type, old_etag, new_etag)
295 """
296 raise NotImplementedError(self.iter_changes)
297
298 def get_comment(self):
299 """Retrieve store comment.
300
301 :return: Comment
302 """
303 raise NotImplementedError(self.get_comment)
304
305 def set_comment(self, comment):
306 """Set comment.
307
308 :param comment: New comment to set
309 """
310 raise NotImplementedError(self.set_comment)
311
312 def destroy(self):
313 """Destroy this store."""
314 raise NotImplementedError(self.destroy)
315
316 def subdirectories(self):
317 """Returns subdirectories to probe for other stores.
318
319 :return: List of names
320 """
321 raise NotImplementedError(self.subdirectories)
322
323
324 class GitStore(Store):
325 """A Store backed by a Git Repository.
326 """
327
328 def __init__(self, repo, ref=b'refs/heads/master',
329 check_for_duplicate_uids=True):
330 super(GitStore, self).__init__()
331 self.ref = ref
332 self.repo = repo
333 # Maps uids to (sha, fname)
334 self._uid_to_fname = {}
335 self._check_for_duplicate_uids = check_for_duplicate_uids
336 # Set of blob ids that have already been scanned
337 self._fname_to_uid = {}
338
339 def __repr__(self):
340 return "%s(%r, ref=%r)" % (type(self).__name__, self.repo, self.ref)
341
342 @property
343 def path(self):
344 return self.repo.path
345
346 def _check_duplicate(self, uid, name, replace_etag):
347 if uid is not None and self._check_for_duplicate_uids:
348 self._scan_uids()
349 try:
350 (existing_name, _) = self._uid_to_fname[uid]
351 except KeyError:
352 pass
353 else:
354 if existing_name != name:
355 raise DuplicateUidError(uid, existing_name, name)
356
357 try:
358 etag = self._get_etag(name)
359 except KeyError:
360 etag = None
361 if replace_etag is not None and etag != replace_etag:
362 raise InvalidETag(name, etag, replace_etag)
363 return etag
364
365 def import_one(self, name, content_type, data, message=None, author=None,
366 replace_etag=None):
367 """Import a single object.
368
369 :param name: name of the object
370 :param content_type: Content type
371 :param data: serialized object as list of bytes
372 :param message: Commit message
373 :param author: Optional author
374 :param replace_etag: optional etag of object to replace
375 :raise InvalidETag: when the name already exists but with different
376 etag
377 :raise DuplicateUidError: when the uid already exists
378 :return: etag
379 """
380 fi = open_by_content_type(data, content_type, self.extra_file_handlers)
381 if name is None:
382 name = str(uuid.uuid4())
383 extension = MIMETYPES.guess_extension(content_type)
384 if extension is not None:
385 name += extension
386 fi.validate()
387 try:
388 uid = fi.get_uid()
389 except (KeyError, NotImplementedError):
390 uid = None
391 self._check_duplicate(uid, name, replace_etag)
392 if message is None:
393 try:
394 old_fi = self.get_file(name, content_type, replace_etag)
395 except KeyError:
396 old_fi = None
397 message = '\n'.join(fi.describe_delta(name, old_fi))
398 etag = self._import_one(name, data, message, author=author)
399 return (name, etag.decode('ascii'))
400
401 def _get_raw(self, name, etag=None):
402 """Get the raw contents of an object.
403
404 :param name: Name of the item
405 :param etag: Optional etag
406 :return: raw contents as chunks
407 """
408 if etag is None:
409 etag = self._get_etag(name)
410 blob = self.repo.object_store[etag.encode('ascii')]
411 return blob.chunked
412
413 def _scan_uids(self):
414 removed = set(self._fname_to_uid.keys())
415 for (name, mode, sha) in self._iterblobs():
416 etag = sha.decode('ascii')
417 if name in removed:
418 removed.remove(name)
419 if (name in self._fname_to_uid and
420 self._fname_to_uid[name][0] == etag):
421 continue
422 blob = self.repo.object_store[sha]
423 fi = open_by_extension(blob.chunked, name,
424 self.extra_file_handlers)
425 try:
426 uid = fi.get_uid()
427 except KeyError:
428 logger.warning('No UID found in file %s', name)
429 uid = None
430 except InvalidFileContents:
431 logging.warning('Unable to parse file %s', name)
432 uid = None
433 except NotImplementedError:
434 # This file type doesn't support UIDs
435 uid = None
436 self._fname_to_uid[name] = (etag, uid)
437 if uid is not None:
438 self._uid_to_fname[uid] = (name, etag)
439 for name in removed:
440 (unused_etag, uid) = self._fname_to_uid[name]
441 if uid is not None:
442 del self._uid_to_fname[uid]
443 del self._fname_to_uid[name]
444
445 def _iterblobs(self, ctag=None):
446 raise NotImplementedError(self._iterblobs)
447
448 def iter_with_etag(self, ctag=None):
449 """Iterate over all items in the store with etag.
450
451 :param ctag: Ctag to iterate for
452 :yield: (name, content_type, etag) tuples
453 """
454 for (name, mode, sha) in self._iterblobs(ctag):
455 (mime_type, _) = MIMETYPES.guess_type(name)
456 if mime_type is None:
457 mime_type = DEFAULT_MIME_TYPE
458 yield (name, mime_type, sha.decode('ascii'))
459
460 @classmethod
461 def create(cls, path):
462 """Create a new store backed by a Git repository on disk.
463
464 :return: A `GitStore`
465 """
466 raise NotImplementedError(cls.create)
467
468 @classmethod
469 def open_from_path(cls, path):
470 """Open a GitStore from a path.
471
472 :param path: Path
473 :return: A `GitStore`
474 """
475 try:
476 return cls.open(dulwich.repo.Repo(path))
477 except dulwich.repo.NotGitRepository:
478 raise NotStoreError(path)
479
480 @classmethod
481 def open(cls, repo):
482 """Open a GitStore given a Repo object.
483
484 :param repo: A Dulwich `Repo`
485 :return: A `GitStore`
486 """
487 if repo.has_index():
488 return TreeGitStore(repo)
489 else:
490 return BareGitStore(repo)
491
492 def get_description(self):
493 """Get extended description.
494
495 :return: repository description as string
496 """
497 desc = self.repo.get_description()
498 if desc is not None:
499 desc = desc.decode(DEFAULT_ENCODING)
500 return desc
501
502 def set_description(self, description):
503 """Set extended description.
504
505 :param description: repository description as string
506 """
507 return self.repo.set_description(description.encode(DEFAULT_ENCODING))
508
509 def set_comment(self, comment):
510 """Set comment.
511
512 :param comment: Comment
513 """
514 config = self.repo.get_config()
515 config.set(b'xandikos', b'comment', comment.encode(DEFAULT_ENCODING))
516 config.write_to_path()
517
518 def get_comment(self):
519 """Get comment.
520
521 :return: Comment
522 """
523 config = self.repo.get_config()
524 try:
525 comment = config.get(b'xandikos', b'comment')
526 except KeyError:
527 return None
528 else:
529 return comment.decode(DEFAULT_ENCODING)
530
531 def get_color(self):
532 """Get color.
533
534 :return: A Color code, or None
535 """
536 config = self.repo.get_config()
537 try:
538 color = config.get(b'xandikos', b'color')
539 except KeyError:
540 return None
541 else:
542 return color.decode(DEFAULT_ENCODING)
543
544 def set_color(self, color):
545 """Set the color code for this store."""
546 config = self.repo.get_config()
547 # Strip leading # to work around
548 # https://github.com/jelmer/dulwich/issues/511
549 # TODO(jelmer): Drop when that bug gets fixed.
550 config.set(
551 b'xandikos', b'color',
552 color.lstrip('#').encode(DEFAULT_ENCODING) if color else b'')
553 config.write_to_path()
554
555 def get_displayname(self):
556 """Get display name.
557
558 :return: The display name, or None if not set
559 """
560 config = self.repo.get_config()
561 try:
562 displayname = config.get(b'xandikos', b'displayname')
563 except KeyError:
564 return None
565 else:
566 return displayname.decode(DEFAULT_ENCODING)
567
568 def set_displayname(self, displayname):
569 """Set the display name.
570
571 :param displayname: New display name
572 """
573 config = self.repo.get_config()
574 config.set(b'xandikos', b'displayname',
575 displayname.encode(DEFAULT_ENCODING))
576 config.write_to_path()
577
578 def set_type(self, store_type):
579 """Set store type.
580
581 :param store_type: New store type (one of VALID_STORE_TYPES)
582 """
583 config = self.repo.get_config()
584 config.set(b'xandikos', b'type', store_type.encode(DEFAULT_ENCODING))
585 config.write_to_path()
586
587 def get_type(self):
588 """Get store type.
589
590 This looks in git config first, then falls back to guessing.
591 """
592 config = self.repo.get_config()
593 try:
594 store_type = config.get(b'xandikos', b'type')
595 except KeyError:
596 return super(GitStore, self).get_type()
597 else:
598 store_type = store_type.decode(DEFAULT_ENCODING)
599 if store_type not in VALID_STORE_TYPES:
600 logging.warning(
601 'Invalid store type %s set for %r.',
602 store_type, self.repo)
603 return store_type
604
605 def iter_changes(self, old_ctag, new_ctag):
606 """Get changes between two versions of this store.
607
608 :param old_ctag: Old ctag (None for empty Store)
609 :param new_ctag: New ctag
610 :return: Iterator over (name, content_type, old_etag, new_etag)
611 """
612 if old_ctag is None:
613 t = Tree()
614 self.repo.object_store.add_object(t)
615 old_ctag = t.id.decode('ascii')
616 previous = {
617 name: (content_type, etag)
618 for (name, content_type, etag) in self.iter_with_etag(old_ctag)
619 }
620 for (name, new_content_type, new_etag) in (
621 self.iter_with_etag(new_ctag)):
622 try:
623 (old_content_type, old_etag) = previous[name]
624 except KeyError:
625 old_etag = None
626 else:
627 assert old_content_type == new_content_type
628 if old_etag != new_etag:
629 yield (name, new_content_type, old_etag, new_etag)
630 if old_etag is not None:
631 del previous[name]
632 for (name, (old_content_type, old_etag)) in previous.items():
633 yield (name, old_content_type, old_etag, None)
634
635 def destroy(self):
636 """Destroy this store."""
637 shutil.rmtree(self.path)
638
639
640 class BareGitStore(GitStore):
641 """A Store backed by a bare git repository."""
642
643 def _get_current_tree(self):
644 try:
645 ref_object = self.repo[self.ref]
646 except KeyError:
647 return Tree()
648 if isinstance(ref_object, Tree):
649 return ref_object
650 else:
651 return self.repo.object_store[ref_object.tree]
652
653 def _get_etag(self, name):
654 tree = self._get_current_tree()
655 name = name.encode(DEFAULT_ENCODING)
656 return tree[name][1].decode('ascii')
657
658 def get_ctag(self):
659 """Return the ctag for this store."""
660 return self._get_current_tree().id.decode('ascii')
661
662 def _iterblobs(self, ctag=None):
663 if ctag is None:
664 tree = self._get_current_tree()
665 else:
666 tree = self.repo.object_store[ctag.encode('ascii')]
667 for (name, mode, sha) in tree.iteritems():
668 name = name.decode(DEFAULT_ENCODING)
669 yield (name, mode, sha)
670
671 @classmethod
672 def create_memory(cls):
673 """Create a new store backed by a memory repository.
674
675 :return: A `GitStore`
676 """
677 return cls(dulwich.repo.MemoryRepo())
678
679 def _commit_tree(self, tree_id, message, author=None):
680 try:
681 committer = self.repo._get_user_identity()
682 except KeyError:
683 committer = _DEFAULT_COMMITTER_IDENTITY
684 return self.repo.do_commit(message=message, tree=tree_id, ref=self.ref,
685 committer=committer, author=author)
686
687 def _import_one(self, name, data, message, author=None):
688 """Import a single object.
689
690 :param name: Optional name of the object
691 :param data: serialized object as bytes
692 :param message: optional commit message
693 :param author: optional author
694 :return: etag
695 """
696 b = Blob()
697 b.chunked = data
698 tree = self._get_current_tree()
699 name_enc = name.encode(DEFAULT_ENCODING)
700 tree[name_enc] = (0o644 | stat.S_IFREG, b.id)
701 self.repo.object_store.add_objects([(tree, ''), (b, name_enc)])
702 self._commit_tree(tree.id, message.encode(DEFAULT_ENCODING),
703 author=author)
704 return b.id
705
706 def delete_one(self, name, message=None, author=None, etag=None):
707 """Delete an item.
708
709 :param name: Filename to delete
710 :param message; Commit message
711 :param author: Optional author to store
712 :param etag: Optional mandatory etag of object to remove
713 :raise NoSuchItem: when the item doesn't exist
714 :raise InvalidETag: If the specified ETag doesn't match the curren
715 """
716 tree = self._get_current_tree()
717 name_enc = name.encode(DEFAULT_ENCODING)
718 try:
719 current_sha = tree[name_enc][1]
720 except KeyError:
721 raise NoSuchItem(name)
722 if etag is not None and current_sha != etag.encode('ascii'):
723 raise InvalidETag(name, etag, current_sha.decode('ascii'))
724 del tree[name_enc]
725 self.repo.object_store.add_objects([(tree, '')])
726 if message is None:
727 fi = open_by_extension(
728 self.repo.object_store[current_sha].chunked, name,
729 self.extra_file_handlers)
730 message = "Delete " + fi.describe(name)
731 self._commit_tree(tree.id, message.encode(DEFAULT_ENCODING),
732 author=author)
733
734 @classmethod
735 def create(cls, path):
736 """Create a new store backed by a Git repository on disk.
737
738 :return: A `GitStore`
739 """
740 os.mkdir(path)
741 return cls(dulwich.repo.Repo.init_bare(path))
742
743 def subdirectories(self):
744 """Returns subdirectories to probe for other stores.
745
746 :return: List of names
747 """
748 # Or perhaps just return all subdirectories but filter out
749 # Git-owned ones?
750 return []
751
752
753 class TreeGitStore(GitStore):
754 """A Store that backs onto a treefull Git repository."""
755
756 @classmethod
757 def create(cls, path, bare=True):
758 """Create a new store backed by a Git repository on disk.
759
760 :return: A `GitStore`
761 """
762 os.mkdir(path)
763 return cls(dulwich.repo.Repo.init(path))
764
765 def _get_etag(self, name):
766 index = self.repo.open_index()
767 name = name.encode(DEFAULT_ENCODING)
768 return index[name].sha.decode('ascii')
769
770 def _commit_tree(self, message, author=None):
771