TDB reading done

meta = {'category': samba, 'date': Wed 24 August 2005}

I’ve now finished the ability to read the various TDB files. I’ve also started on the “mapping” LDB module that will allow a simple mapping from one objectclass to another. We’ll need this for the backwards compatible LDAP backend for Samba3.

This backend will never be used itself but instead have other backends based on it (e.g. ldb_samba3 for a Samba3-LDAP-to-samdb backend)

Each backend that uses this skeleton has to specify an array of struct objectclass_mapping’s (see ldb_map.h).

If no mappings are defined for an objectclass, it will be kept as is.

Will also keep cache of mapped dn’s so we don’t have to traverse the tree each time

So, TODO left:

• finish ldb_map
• fill ldb_samba3’s mapping table
• support smb.conf parsing in lib/samba3/
• support generating LDIF’s

comments.

Status Update

meta = {'category': samba, 'date': Tue 23 August 2005}

My Summer of Code project is coming along nicely (although I haven’t been very actively blogging about it). I’ll be working on full-time until the deadline, I guess, but I think I’ll make it.

So far I’ve got the following done:

• Reading of tdbsam
• Reading of policy database (still need to add privilege support)
• Reading of idmap database
• Reading of registry file (still needs some work)
• Reading of old smb.conf files
• Reading of group mapping database
• Initial work on ldb_map backend

The major lump of work now is the ldb_map backend and the conversion of the smb.conf.

comments.

Status Update

meta = {'category': samba, 'date': Tue 23 August 2005}

Althought I haven’t been writing my weekly Samba4 status updates for a while now, development on Samba4 certainly hasn’t been stalled. There have been a lot of improvements since June. I’ll try to sum up most of them here, hopefully not forgetting too much:

• ldb_sqlite is done afaik, still need to try it
• Deryck and Tridge have been working on SWAT, giving it a user-friendly interface
• Lots of progress in the directory synchronisation interface by Brad (Summer of Code participant) and Metze
• Andrew has done more work on Kerberos integration such as the kpassword protocol and the PAC
• Yet more LDB updates from Simo, not sure what exactly :-)
• Quite a couple of things have been converted to Javascript. ejs is now being used for some of the testing as well as the provisioning script. Mostly by Tridge and Mimir
• pidl updates and build system updates from me

… and much, much, more

I’m personally still a bit skeptical on using ejs for things other then SWAT, but I can’t really describe why. Perhaps it’s the fact that when I used one of its early incarnations, Javascript was very hairy to use. I also don’t get that “warm fuzzy feeling” when I use Javascript that I get when I use C, C#, Haskell or Python, but I guess that’s not really a good argument. :-) The C interfaces to EJS are certainly very easy to use and EJS is a nice small library.

CIFS was good again this year. Samba.org has some pictures.

comments.

Ethereal parser generators

meta = {'category': samba, 'date': Tue 23 August 2005}

Together with a few others (Tim Potter, Alan De Kok) I have been working on parser generators in pidl for DCE/RPC interfaces over the last one-and-a-half years. However, none of our attempts got far enough to generate both clean and working code.

I sat down with Ronnie Sahlberg from Ethereal at the CIFS 2005 Conference and discussed some of the approaches he had taken in his idl2eth compiler (which is a big hack, but does generate nice code). With the help of Ronnie I was able to write a new module in about a day, taking to account some of the concerns the ethereal developers have.

See the new Parse::Pidl::Ethereal::NDR module for details. Hopefully Ethereal will adapt pidl with this new module for generating at least some of the DCE/RPC parser generators during the next month or so.

comments.

Participating in the Google Summer of Code!

meta = {'category': samba, 'date': Fri 15 July 2005}

This summer, I will be participating in the Google Summer of Code. The project I will be working on is the Samba3->Samba4 upgrade path.

In my original proposal, I split this project up into the following subtasks:

• Each of the various password database backends (tdb, ldap, smbpaswd) available in Samba3 need to be accessible and converted to the SAM database in Samba4.
• Printer database needs to be read from the Samba3 TDB and added to the apprioprate LDAP subtree.
• The WINS database needs to be converted from both plain text file and TDB file to LDAP/LDB.
• The account policy database needs to be read in from a TDB and applied to the data in LDAP/LDB.
• The privilege database needs to be read from a TDB and added to the SAM database in LDB.
• Group mappings need to be read from the TDB and added to the SAM database.
• The share info database and the configuration file from Samba3 need to be read and converted to either xattrs or a TDB in Samba4
• Secrets.tdb, containing the domain formation, needs to be merged into the SAM database.
• Last, but not least, a Samba3 configuration file should be parsable in Samba4. This file can be used for producing a Samba4 smb.conf file as well as for updating the various databases mentioned above.

I will probably have to change this later on, but it’s a good list to start with.

comments.

Samba4 this week: Bug squashing

meta = {'category': samba, 'date': Sun 19 June 2005}

It has been a relatively quiet week for Samba4. Tridge has been working on supporting some more of the Windows LDAP (such as binary or and and operators). Andrew has merged the configuration loading and the command-line handling.

Most changes were bugfixes for existing code by Tridge, Andrew, Metze, Simo, Rafal and me.

comments.

Samba4 this week: “make test”, EJS Wrapper scripts, TDR and strings in pidl

meta = {'category': samba, 'date': Mon 13 June 2005}

It has been another interesting week in Samba4-land. We have now enabled the Samba4 test suite on all machines in the Samba buildfarm. I did the original (small) patch, metze has done a lot of nice work cleaning up the output, making sure it didn’t try to run the tests when smbd was down, etc and tridge has added options to smbd and smbtorture to make sure they don’t get stuck in an endless loop.

The SPOOLSS test now also works, thanks to continuing efforts from Metze over the past few weeks.

HotaruT and Derell have been working on, respectively, a PostGreSQL and a SQLite backend to LDB. I haven’t checked them out yet (they’re still mostly works-in-progress), but it’d be interesting to see whether they can beat TDB in performance.

I’m currently looking at implementing a new (very simple) data representation format in pidl, so that we don’t have to depend on NDR for, for example, the NetBIOS implementation. It’ll be a very simple format, not supporting pointers and stuff like that, so probably a bit more efficient as well.

Another thing I’m looking at is “proper” string support, e.g. similar to what MIDL has. Pidl’s current extensions (automatic conversion to UTF8, etc) could be supported using a couple of custom attributes.

Other changes this week included improvement of IRPC performance by tridge, EJS improvements by tridge and Tim, more heimdal fixes by Andrew Bartlett.

comments.

Rewriting pidl in Haskell

meta = {'category': samba, 'date': Tue 07 June 2005}

Now that my rewrite of pidl has been finished, I am wondering what the future for pidl will be.

Looking at the new structure of the source code, I notice that pidl’s structure is now very simple. First, it generates a datastructure representing the IDL file; then it generates new datastructures by walking the first one recursively. In the last step, it recursively walks a datastructure generating output.

Things like this are very easy to do in Haskell, using a trivial parser and a couple of fold functions, as I learned at the course on grammars and parsing at uni.

The parsing step is probably the most difficult one, i.e. going from a file to something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

type Property = (String, [String])

type EnumMember = (String, String)

data Type = Struct (Maybe String) [DataMember]
             | Union (Maybe String) [Maybe DataMember]
             | Enum (Maybe String) [EnumMember]
             | Reference String
             | Interface String (Maybe String) [Property] [InterfaceMember]

type Typedef = (String, Type)

data InterfaceMember = Function String [Property] Type [DataMember]
                     | Typedef String [Property] Type
                     | Const Type String String
                     | Type Type

type DataMember = (Type, Int, String, [Property], [String])

Ah well, back to more important things now…

comments.

Management and status mechanisms + KDC updates

meta = {'category': samba, 'date': Sun 05 June 2005}

Samba4 development is still progressing nicely. Tridge has been working on a interprocess communication mechanism for managing and gathering statistics from smbd processes, called irpc.

Andrew Bartlett has done more work on the Samba4 KDC. A lot of work is now going by both Andrews to support an in-tree build of heimdal, using Samba4’s build system.

Metze and I have been doing small fixes and improvements to the build system and pidl.

comments.

More Samba4 progress - pidl2, libkdc and SWAT

meta = {'category': samba, 'date': Thu 26 May 2005}

A lot has happened in the Samba4 world over the last week.

Andrew Bartlett has got a simple domain join to a Samba DC using kerberos working using a hacked-up version of Heimdal kerberos. He’s now looking at the best way of doing a KDC for Samba4 - either writing one from scratch, borrowing one from one of the existing open source implementations or borrowing parts of one of the existing ones (a “libkdc”). There’s been some heated discussion about this on the related mailing lists.

I have merged pidl2 (my rewrite of pidl, our IDL compiler) into the main Samba4 SVN branch. This is something I have been working on for the past 2 months.

Tridge has started working on Samba 4’s internal web server, which will be used for the next generation of SWAT. The webserver supports (a subset of) javascript on the server-side scripting, thru the AppWeb Embedded JavaScript library.

comments.

cldap server working

meta = {'category': samba, 'date': Wed 18 May 2005}

Tridge just got the cldap server working! That means we’re another item down before we can release an alpha.

I’m still stuck working on the rewritten version of pidl as allocation issues are causing me headaches.

comments.

dissecting WMI

meta = {'category': samba, 'date': Tue 12 April 2005}

Over the past few days, I have been busy trying to figure out what WMI is and how we could possibly implement it in Samba.

WMI is Microsofts’ implementation of the WBEM standard; unlike most other vendors, it however does not use the CIM-XML but DCOM.

I did a quick capture of the network traffic when running the following script:

1
2
3
4
5
6
7
8
9

For Each Host In WScript.Arguments
   Set WMIservice = GetObject("winmgmts:{impersonationLevel=impersonate}!\\\\" & host & "\root\cimv2")

   Set colsettings = WMIservice.ExecQuery("SELECT * FROM Win32_Processor")

   For Each proc In colsettings
       Wscript.Echo(host & ": " & proc.description)
   Next
 Next

It appears WMI uses just the very simple set of DCOM interfaces (IOXIDResolver, IRemUnknown2, IRemUnknown, ISystemActivator) combined with a small number of WMI-specific interfaces.

This means that implementing the client-side of WMI will be relatively easy - once I get DCOM working fully again, it should just be a matter of adding the ODL for the required interfaces.

comments.

Mixing pointers and arrays in IDL

meta = {'category': samba, 'date': Mon 28 March 2005}

There is a very thin line between the way various encodings that can be given to combinations of arrays and pointers in IDL files. Consider the following examples:

[size_is(…)] int data[] - Inline array of integers [size_is(…)] int *data - Pointer to array of integers [size_is(…)] int *data[] - Array of pointers followed by array of integers [size_is(,…)] int *data[] - Pointer to an array of integers

Summarizing the meaning of the operators:

• [] stands for an in-line array
• * is a pointer, possibly to an array

comments.

DCOM client support

meta = {'category': samba, 'date': Mon 08 November 2004}

DCOM client support finally works! TODO things left for DCOM now are:

• Server side support
• Supporting base interfaces outside current file
• Fix annoying bug in pidl WRT to IStream_Read
• Implement IUnknown
• Implement pinging

comments.

My Samba 4 TODO-list

meta = {'category': samba, 'date': Sun 07 November 2004}

It seems the list is getting longer rather then shorter…

• finish gepdump (MGMT support)
• finish DCOM - generate <code>dcerpc_pipes</code> table dynamically - allow binding strings to contain no transport name
• client_credentials struct
• more extensive support for expressions in pidl
• build system in 100% perl

comments.