University of Washington

1. University of Washington

2. Why knowing LDAP will help you manage & troubleshoot your ADDisclaimer: We won’t talk much about LDAP as a protocol or API (I’m anticipating huge sighs of relief); instead we’ll focus on the directory part of LDAP & how this knowledge might be useful in the context of AD.

3. Who am I? • Software Engineer at Univ. of Washington • ~2000 managed workstations ** • Large forest (~40 domains/90 DCs) • Helped design Stanford’s AD deployment ** • Help manage the windows-hied mailing list ** • Wrote LDAP Directories Explained for Addison-Wesley—intro to LDAP **

4. Premise Microsoft has done a better job integrating LDAP into the user experience than any other directory product. But nearly-seamless integration doesn’t mean you should ignore the technology. Knowledge empowers understanding. If you understand the technology behind a product, then you can understand the product and it’s behavior better. **

5. Overview • LDAP basics & a little about how it underlies all AD tools & APIs • LDAP search filters can let you quickly grab info—a quick primer • The forgotten partition: the config partition is central to many key operations. • Some things can't be done w/o LDAP; there aren't MS tools to do what you want. • When things go wrong, be prepared for sub-optimal PSS solutions. • Distributed directories and powerful AD functionalities like persistent searches. • Understand what major product upgrades are doing behind the scenes. • Windows 2003 “forest prep” and “domain prep” • VBS Scripting example • AD LDAP strengths & a wish list

6. Directory basics • LDAP adopts the X.500 directory standards. (this adoption is mostly implicit which has lead to some problems in the schema assumptions). • This means that a LDAP directory is a tree structure, where any given child entry only has one parent; no web-like structure. • An axiom of this structure is that the name of an entry also gives you it’s location. This is a powerful feature of the design that isn’t often recognized. • A directory is really just a special purpose “fast, read-oriented” DB. You wouldn’t use a directory for a modification-intensive purpose—you’d use a DB. The converse axiom seems to be less recognized; too many people (or should I say products?) use a database for a low-mod/hi-read purpose when a directory would work better. • As with ALL protocols, there are special characters that get interpreted in special ways when using that protocol. So you have to be careful to encode those characters (escape with a \). So if I wanted to search the location cn=C&C,dc=endor,dc=sw—I’d better ask for cn=C\&C,dc=endor,dc=sw. The converse axiom is to avoid special characters in attribute values as much as possible. You should also be sure that your LDAP vendor knows about buffer overflow & command injection schemes that make use of these special characters. **

7. Basics (2) • The 10 LDAP operations are: • Bind • Search • Compare • Add • Delete • Modify • ModifyRDN (rename) • Unbind • Abandon • Extended These operations are standardized, i.e. every LDAP server & client must support them in the same way. Further, every LDAP API supports them the same way, with a common data structure Controls modify these operations to create a new action (e.g. server-side-sort). Controls aren’t mandatory. Options modify all the operations in a session (e.g. SSL). Options aren’t mandatory. There are tons of new IETF proposals for extensions to LDAP all the time. Many of the most interesting proposals have gotten implemented in AD. ** We’ll see more on AD specifics of controls later.

8. Basics (3)

9. LDAP underlies most tools • ADSI uses LDAP • AD MMC tools use LDAP • Exchange uses LDAP • AD reskit tools use LDAP … • If you want to access data in AD, you are using LDAP. **

10. Finding a directory entry—or a set of entries • A LDAP search involves 3 mandatory parameters: • A base DN, i.e. where to start • Ex: dc=mydomain,dc=myschool,dc=edu • The scope, i.e. how far to look • Subtree=the base DN & all entries below • One=all entries at same level as base DN • Base=Just base DN • A search filter, i.e. what to look for • Search filters are a composition of statements, which are surrounded by parentheses • A statement has 3 elements: • Attribute type, e.g. objectclass or mail • Comparison operator e.g. = or >= • Attribute value – Wildcards are possible to match zero or more characters • Combination of statements is possible with filter operators which precede the statement to form a composite statement (with parentheses) • &=AND |=OR !=NOT • Ex: (|(objectclass=user)(objectclass=computer)) i.e. the user or computer entries ** • Ex: (&(objectclass=user)(cn=a*)) i.e. the user entries with names that begin with a ** • Note that the common GUI tools that are built on top of LDAP either assume this info or have you select it (while ignoring the optional parameters). They also make use of something called Display Specifiers to get a GUI look, but that’s outside this talk. **

11. LDAP Search (part 2) • LDAP search optional parameters • What attributes to return • Blank = all (operational left out) • 1.1 = none (1.1 is a special OID) ** • Attrib1,attrib2,etc for a limited list (this is only way to get operational attribs) • derefAliases (AD doesn’t support aliases!!) ** • neverDerefAliases=don’t follow aliases • derefInSearching=lookup all except baseobject • derefFindingBaseObj=lookup only baseobject • derefAlways=lookup all • sizeLimit = number of entries returned • 0 is default & means give me all results • May be limited by configuration (which AD sets to 1000) • If more are found, this is noted in response. • timeLimit = Max time (in seconds) to search • 0 is default & means give me all results • May be limited by configuration (which AD sets to 120) • If more are found, this is noted in response. • typesOnly ** • True = lists only attribute types, i.e. sn,cn, etc. • False (default) = lists both attribute type and value, i.e. sn=Arkills, cn=Brian Arkills, etc.

12. LDAP Clients • The Search Assistant/Address Book application hides the complexity of LDAP search filters nicely. This means users don’t need to know LDAP. ** • BUT …. if you need the search filter to behave differently, then it isn’t necessarily easy to change. You will want to check out a few places: • Q311829 describes how to turn off displayname searching. • Mapisvc.inf in the [EMABLT] section has the default search filter used by the Address Book. You can modify it. By default it uses: (&(mail=*)(|(mail=%s*)(|(cn=%s*)(|(sn=%s*)(givenName=%s*)))))i.e. Must have mail attrib set and have your search string (with wildcard afterward) in the cn, sn, mail, or givenName attribs • MMC ADUC has some configurable parameters—check out the View menu>Filter Options ** • ADSIEdit (support tools) is nice for single operation use, but slower than ldp.exe. I use it for 3 things: schema consultation, for the few operational attribs ldp doesn't permit, & the security GUI so I don't have to decode the ntsecuritydescriptor. **

13. The forgotten partition • The configuration partition is central to many key operations. Viewing its data directly via ldp or adsiedit instead of one of the other dumbed-down MS tools is very enlightening, and gives you much more control. ** • Examples of critical data in the config partition: • Cn=ExtendedRights holds special rights (which you can create for use in your app). • Cn=Partitions holds a list of all the known partitions in the forest. This is useful for a number of forest-wide search scripts. Beware crossref entries that get mangled; at UW we’ve seen it 3 times—more on this later. • Cn=Services holds a variety of configuration data for network services. Most notable are the LDAP administrative parameters such as the maximum page size, and a default forest-wide policy for these settings (more later). And Exchange lives here--more on that soon • Cn=Sites holds the AD replication topology. An entry for every domain controller is here; this is useful for forest-wide DC search scripts. • Cn=WellKnownSecurityPrincipals holds the built-in user & group accounts like Authenticated Users. Dunno if you can create your own … but w2k3 creates new ones, so I’m guessing you can.

14. AD LDAP configuration parameters • A default forest-wide policy is at cn=Default Query Policy,cn=Query-Policies,cn=Directory Service,cn=Windows NT,cn=Services,CN=Configuration,DC=Mycompany,DC=com. The lDAPAdminLimits attribute stores the below config settings in a string format. Additionally, the lDAPIPDenyList attribute stores any IPs (can use subnet mask) that should be denied connections. ** • Additional policies can be created and linked to specific domain controllers, so individual domain controllers have settings different from the default. • Types of parameters you can set: • MaxPageSize: largest size (1000) • MaxActiveQueries: max queries from all clients (20) • InitRecvTimeout: timeout between two LDAP servers (120) • MaxConnections: max client connections (5000) • MaxConnIdleTime: client timeout (900) • MaxNotficationsPerConn: max number of psearch per client (5) • MaxQueryDuration: max time on one operation (120) • MaxResultSetSize: max size of data (262144) • MaxTempTableSize: max temp. table used for sorted search (10000) • MaxPoolThreads: number of directory threads (4) • Ntdsutil also offers a way to modify these. **

15. Exchange lives in the config partition • Critical data it stores here includes: • Address book definitions, i.e. which accounts are displayed • Where RUS lives for each domain partition • SMTP connections • Recipient Policies • Admin Group specific settings, e.g. store/MTA/SA parameters, supported protocols, Exch SP level, etc. • An example … how Exchange forms the list of accounts that are displayed in the Global Address List isn't documented anywhere publicly, but it is a functionality you can modify via LDAP. The default value is: purportedSearch=(&(mailnickname=*)(|(&(objectCategory=person)(objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=user)(|(homeMDB=*)(msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=contact)(objectCategory=group)(objectCategory=publicFolder))) Complex looking, huh?

16. Exchange (part 2) With a little reformatting for better readability: purportedSearch=(&(mailnickname=*)(| (&(objectCategory=person)(objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=user) (|(homeMDB=*)(msExchHomeServerName=*))) (&(objectCategory=person)(objectClass=contact)(objectCategory=group)(objectCategory=publicFolder) )) In a slightly simplified statement, this equates to Gotta be mail-enabled and either a. person+user (with useless additions) b. person+contact c. group d. public folder You can modify this definition to further restrict the GAL (for privacy) or modify other address lists or recipient policies that have a purportedSearch attribute (and circumvent the e2k GUI field limit at the same time).

17. Left without a MS tool • Like the last example, there are some things that can't be done without direct LDAP manipulation; there aren't high-level tools to do what you want. • For example, what do you do when you need to manually delete a DC entry in the config partition (so the replication topology is clean) when the domain naming context (i.e. entry under cn=partitions) is gone? • ntdsutil can't manually remove a dead DC with no NC; it gives an error. You might be able to do a metadirectory cleanup, but that requires a reboot and time. And those two tools don’t really have less obscure syntax than LDAP, so it’s far easier to just delete the entry via LDAP, so why bother with ntdsutil? Well … I’ve been told that ntdsutil in some cases does some extra non-LDAP stuff to the DB cross-linked entries behind the curtain. But I’ve never seen this make any difference in practice. • What do you do when you need to manually delete a domain, but the domain is still running? • You delete the domain trust entry in the forest root domain, the domain partition entry in the config partition, and any DCs in the config partition. If you don’t delete the trust, the other entries will come back from the running domain.  • I’m sure there are other examples—I just haven’t run into them yet.

18. Troubleshooting & being prepared for the MS engineer Earlier, I mentioned a crossref mangling issue. We had a problem in our forest where replication was failing to several domains. Most (but not all) of the partners of those domains would complain on the replication trigger about an invalid SRV record. The SRV record for the DCs in those domains was registered. Looking more closely at the event descriptions, I noticed that the SRV record noted had a mangled name with a tell-tale collision-style prefix (usually CNF). Where was this coming from? I used a LDAP search to find all the collision records in AD, and found some suspicious values on the partition entry for those domains (which had mangled values only on some of the DCs in our forest). Turns out that the problem was caused by a directory collision on the object representing the domain partition. The collision probably happened when the domain was demoted, then recreated prior to full replication throughout the forest. After some extensive troubleshooting (and trying to manually change those mangled values), we got PSS involved. Initially, they thought a hotfix would have to be applied to EVERY domain controller in our forest, plus we’d need to run a special version of ntdsutil in DS Restore Mode on each DC. With ~90 DCs geographically dispersed across several campuses, we were not thrilled with pursuing that solution. I went into the case with the conviction that we simply needed to modify the attribute value that had been mangled, and the change would be replicated. This attribute was system-only, meaning that only AD itself can modify it (not to be confused with the local system account). I knew that every other LDAP product had a way to modify system-only attribute values. So I remained steadfast through 4 levels of engineers that MS must have a way to modify system-only data. I ended up with the fellow who wrote repadmin—nice guy. And they did … flip a registry bit, modify the value, replicate, and flip the registry bit back. I’ve heard that MS doesn’t want folks to know how to modify system-only data. I can imagine that some basic architectural assumptions might be subverted if that knowledge was widespread—although only by those administrators who already have the ACLs and own the forest root domain. Maybe MS is nervous about AD being modified to work better with products they are in direct competition with. Whatever the case, I still have never seen this documented by MS.

19. Distributed Directories • You likely have more than one directory—many of which are databases, even though they don’t have a high volume of modification. You’ve probably got HR, students, ID cards, NetID, alumni, etc. • Unifying distributed directories is about how to make all these disparate source systems come together. Some folks call this the metadirectory problem, but that is only one of several possible solutions.

20. Distributed directories (2) • As you try to fit two or more “directories” together, the issues include: • Protocol: what do they talk? LDAP is nice because it has universal clients. Often LDIF is used--although I think DSML will replace LDIF. ** • Schema design: I call the department “department”, but you call it “dept”. How do we reconcile the two “labels” (attributes)? Hard problem … more on this later. • Data consistency: I call our department “C&C”, you call it “Computing and Communications”, and he calls it “CAC”. How do we reconcile the multiple values (attribute values)? Also a hard problem … more on this later. • Data Structure: The way data is structured can affect the usability of that data. And different directories will structure the data in the way that most benefits its own purposes. So how do you reconcile different structures where the data should have close relationships (like the NetID to the respective people systems)?

21. Distributed directories (3) • Solutions include: • Metadirectories—hardest to implement because it must tackle all the issues. Usually a “connector” or “harvester” sucks data into a middle system, scrubs it to compliance with scripts, then publishes it (or returns it to the original systems). There are both commercial & homegrown solutions e.g. Microsoft Metadirectory Services or Stanford’s Registry Harvester. • Loosely connected directories—easiest to implement, but requires LDAP or X.500 directories and it doesn’t solve big discrepancies to any great satisfaction. You use referrals to paste namespaces together. A client can connect to any of the systems and get seamless referrals to the others. • Client portal or middleware service—fix the discrepancies and bring together the data in a website where clients have a limited subset of queries. This has many of the same problems to tackle as a metadirectory (plus UI design), but when complete it isn’t as useful. For example, you can’t use it to populate a network OS directory like AD, whereas a metadirectory would work.

22. Advanced AD functionality • Controls allow you to modify the standard LDAP operations with a new behavior. Common examples of controls are the sorted search (sorted results) and the paged search (usually to bypass size limits). AD supports the most controls of any LDAP server product. • Persistent Search functionality is very useful for metadirectory synchronization. The basic idea is to allow a search that functions over a time period, instead of requiring repetitive searches. AD supports • dirsync, “give me all the changes since a point in time (based on a USN)”, • psearch, “give me all the changes from now on until I cancel the operation”. • Tree delete—allows deletion of a container and children. Not widely supported. • Lazy commit—allows batch changes to be queued to optimize performance. • Virtual List View—allows search to specify sort order, the number, and the position in the results. Useful for “scrolling” behavior when the result set is large. http://msdn.microsoft.com/library/en-us/netdir/ad/polling_for_changes_using_usnchanged.asp • Referrals allow you to extend your directory to other LDAP servers. There are 2 types of referrals: a default referral or a referral entry. A default referral is returned on any search that asks for something outside the server’s naming contexts. A referral entry is an actual directory entry that can be returned to the client on any search. It redirects the client to another directory space, and by default the client executes the same search on that namespace too. You end up w/ the composite of the two results. The bottom line is that you can create a distributed directory that spans multiple implementations.

23. Windows 2003 forestprep %systemroot%\System32\Debug\Adprep\Logs\Latest_log folder • Schema changes: • 49 objectclasses added. (was 142 or 300 w/e2k) • 208 attributes added. (was 863 or 1716 w/e2k) (now I can store my favorite drink) • 17 Extended rights added. (was 48) • 9 syntaxes added. (was 18) • Inetorgperson is finally added; special actions to take if you added it & e2k already. • Other changes! (not documented): • ACL modification to support new features on: • cn=sites,cn=configuration • cn=configuration • cn=schema • Sam-domain objectclass • Computer objectclass • Other less notable entries • Dunno what any of the changes are as they aren’t logged (I didn’t have extended time to compare & contrast to another non-forestprep’ed forest)

24. Windows 2003 domainprep MS documentation incorrectly states that domainprep just verifies that forestprep has run. What it does: • Verifies that forestprep was run • Creates a bunch of directory entries to support new functionality (such as the WMIPolicy, Program data, & IP Security entries) • Makes ACL & attribute changes to the root of your domain & 8 other locations. These amount to providing new extended rights support. • Adds the anonymous logon SID to the pre-windows 2000 group • Makes ACL changes to GP directory entries & their container • Adds 4 new SID prefixes to the foreignsecurityprincipals container (I've always wondered about this functionality ... anyone know more?): • S-1-5-11 <Authenticated User> • S-1-5-20 <Network Service> • S-1-5-4 <Interactive> • S-1-5-9 <Server Logon> S-1-1-0 <Everyone> was already there.

25. Scripting (using LDAP via ADO) There are lots of other scripting/programming resources you can use with AD. ADSI is an obvious one. Microsoft also has a Windows LDAP API. I’ve chosen to demo a VBS using ADO to access an ADSI provider as it is a very accessible option for those of us who haven’t scripted much. We’ll look at an basic-to-intermediate set of scripts to illustrate several of the facts I’ve noted. I’ll use two different ADO-ADSI dialects to illustrate the diversity of functionality. Sidfilterwrapper: will find all domains in forest, find a DC for each domain, scan that domain for external trusts, then for each external trust scan for sidfiltering. I’ve omitted the code for the wrapper to save time.

26. Scripting (part 2) • List domains in forest Set rs = CreateObject("ADODB.Recordset") Set con = CreateObject("ADODB.Connection") Set com = CreateObject("ADODB.Command") strContext = "GC://dc=endor,dc=sw" con.Provider = "ADsDSOObject" con.Open ("DS Query") Set com.ActiveConnection = con com.CommandText = "Select name FROM '" & strContext & "' WHERE objectclass = 'trusteddomain'" Set rs = com.Execute rs.MoveFirst While Not rs.EOF wscript.echo rs.Fields("name").Value rs.MoveNext Wend

27. Scripting (part 3) • Find DCs for domain domain = wScript.Arguments.Item(0) Set rs = CreateObject("ADODB.Recordset") Set con = CreateObject("ADODB.Connection") Set com = CreateObject("ADODB.Command") strContext = "GC://cn=configuration,dc=endor,dc=sw" strLDAPfilter = "(&(objectclass=server)(dnsHostName=*." & domain & "))" strLDAPattr = "dnsHostName" strLDAPscope = "subtree" strLDAP = "<" & strContext & ">;" & strLDAPfilter & ";" & strLDAPattr & ";" & strLDAPscope con.Provider = "ADsDSOObject" con.Open ("DS Query") Set com.ActiveConnection = con com.CommandText = strLDAP Set rs = com.Execute rs.MoveFirst While Not rs.EOF if split(rs.Fields(0).Value, "." & domain)(0) = split(rs.Fields(0).Value, ".")(0) then wScript.Echo rs.Fields(0).Value end if rs.MoveNext Wend

28. Scripting (part 4) • Scan domain for external trusts: arg0= “LDAP://” & outputOfLastScript –can’t use GC ‘cuz trust attribs aren’t there strContext = wScript.Arguments.Item(0) Set rs = CreateObject("ADODB.Recordset") Set con = CreateObject("ADODB.Connection") Set com = CreateObject("ADODB.Command") con.Provider = "ADsDSOObject" con.Open ("DS Query") Set com.ActiveConnection = con com.CommandText = "Select name,trustdirection,trusttype FROM '" & strContext & "' WHERE objectclass = 'trusteddomain'" Set rs = com.Execute rs.MoveFirst While Not rs.EOF If rs.Fields("trusttype").Value = 1 then If rs.Fields("trustdirection").Value > 1 then wscript.echo rs.Fields("name").Value End If End If rs.MoveNext Wend

29. Scripting (part 5) • Scan for sidfilter forestdomain = wScript.Arguments.Item(0) externaldomain = wScript.Arguments.Item(1) Set wshShell = CreateObject("wScript.Shell") wshShell.CurrentDirectory = "c:\Program Files\support tools" sCommandpath = "netdom.exe" sCommand = sCommandpath & " trust " & forestdomain & " /D:" & externaldomain & " /filtersids" set WshExec = wshShell.Exec(sCommand) sStdOut = WshExec.StdOut.ReadAll If instr(lcase(sStdOut),"filtering is not enabled") > 0 then wScript.Echo "None" Else wScript.Echo "Enabled" End If

30. AD as a LDAP Product • Microsoft has *great* client integration but could give better customization. • Microsoft has superior advanced features (controls, replication, etc.), with only Directory Server (netscape/sun) in the same league • With AD/AM they’ve shed the biggest knock against them, i.e. the requirement to run a network OS in order to use their LDAP server.

31. Brian’s AD Wish List • Reduce number of directory collisions or at least raise the visibility of collisions to a higher level (like logging them with a warning event!). The algorithm AD uses is extensive & fancy, but it doesn’t do the job well enough. And the tools for fixing collisions aren’t sophisticated at all. • Fix non-Internet standard schema mistakes. You can’t claim great integration w/o following standards. • sn is single-valued; should be multi-valued • Alias object class support. It’s implied as part of the LDAP standard. Every LDAP server except AD supports it. Would allow institutions like HiEd to have dummy entries for people in multiple locations, helping with the “multiple affiliations--which OU does the account go into?” problem. • Dynamic access control like Directory Server (either Netscape or Sun—used to be iPlanet) i.e. ability to set an ACE that changes the access depending on the value of an attribute on the target entry or the user’s entry. • Extended match filter support. This would allow searching on the DN of an entry. • Implement the LDAP standards wrt what characters are considered special & in need of being escaped. Document what customers should do with respect to this. • Chaining support. This would mean that AD could be configured to chase referrals instead of the client. This would pave the way for Global (or not so global) catalog functionality with cross-forest trusts. • Improve usability of ldp.exe: • Some operational attributes won’t display. Why? • Smart/auto fill-in would be helpful for typing long x500 strings. • Make results window focus by default on results of last operation. • AD on a non-Windows platform 

32. University of Washington