.NET Framework - powershell memory problem agianst AD with quest (get-qaduser)

Asked By Did on 16-Jul-07 09:27 AM
Hi all,
While I'm trying the enumarate my 30000 users with quest get-qaduser
(use -sizelimit 0 to remove limit of 1000 users, the command run from
script) I receive the following error:

Get-QADUser : Exception retrieving members: "Exception of type
'System.OutOfMemoryException' was thrown."
At C:\scripts\police\monad\UsersAD_ver3.ps1:29 char:21
+ $users = get-QADUser  <<<< -IncludeAllProperties -SearchRoot
'prod.police.dom/Users' -SizeLimit 0

I'm running my script from strong server with 3 giga in RAM.

this is my ps1 script:

$users = get-QADUser -IncludeAllProperties -SearchRoot 'myDomian.dom/
Users' -SizeLimit 0

foreach ($user in $users)
{
write-host "Working now on user " $user.SamAccountName
$proxy = $user.proxyAddresses
foreach ($prox In $proxy)
{
[string]$strProx = $prox
if ($strProx.contains("SMTP"))
{
write-host $strProx
$File_Title = $user.SamAccountName + ";" + "$strProx"
out-file -filepath C:\scripts\police\monad
\UsersToMarina.log -append                            -inputobject
$File_Title -encoding ASCII -width 50
}
}#end foreach proxy
}#end foreach user



Any suggestions ?




oMV replied on 16-Jul-07 11:02 AM
can you test if you have the same problem using native powershell as I
successful used it on 100.000 + users

You can also use the native PowerShell method for this that works fine:


Function Get-AdUser ($name) {
(New-Object
System.DirectoryServices.DirectorySearcher("(&(CN=$name)(objectClass=User))­")).findone().getdirectoryEntry()



}

Greetings /\/\o\/\/
oMV replied on 16-Jul-07 11:06 AM
Oops, forgot pagesize in example :

Function Get-AdUser ($name) {
$ds = New-Object
System.DirectoryServices.DirectorySearcher("(&(CN=$name)(objectClass=User))")
$ds.PageSize = 900
$ds.findAll().getdirectoryEntry()
}

Greetings /\/\o\/\/
Brandon Shell replied on 16-Jul-07 11:24 AM
This should be

function Get-AdUser ($name) {
$ds = New-Object
System.DirectoryServices.DirectorySearcher("(&(CN=$name)(objectClass=User))")
$ds.PageSize = 900
$ds.findAll() | %{$_}
}

Result
52# Get-aduser user1

Path
Properties
----
----------
LDAP://CN=user1,CN=Users,DC=test,DC=domain, ... {samaccounttype, lastlogon,
lastlogontimestamp, profilepath,

Will post results on large number of users shortly.
Oisin Grehan replied on 16-Jul-07 12:00 PM
Instead of storing 300,000 user objects at once in the $users
variable, try using a pipeline instead:

get-QADUser -IncludeAllProperties -SearchRoot 'myDomian.dom Users' -
SizeLimit 0 | % { ... process $_ (a user) ... }

I'm not familiar enough with the internals of powershell to know
whether this technique is just an illusion, but logically speaking, I
would say that it is less stressful on memory this way...

- Oisin
/\\/\\o\\/\\/ [MVP] replied on 16-Jul-07 01:28 PM
I tested this with 200.000 + users (corrected version as Brandon did post
(to quick modifing the code as it was using findone() before and did return
the DirectoryObject ), but  I used the pipeline to process, and output

sorry, did miss the variable assignement on provided sample .
as oisin did explain also, if ou handle lot's of data do as much processing
you can in the pipeline,  also on big queries, same as with WMI you have to
be carefull to select only the properties you need. and do not use commands
that block the pipeline like sort, always do your processing on the ADSI
side.

Greetings /\/\o\/\/
oMV replied on 16-Jul-07 02:06 PM
Brandon is still testing a bit together with me ( no big domain handy here )
currently we still have a test running returning all properties

but this optimized version should make a big difference and I think will
work in your case  :

Function Get-AllAdUsers {
$ds = New-Object
System.DirectoryServices.DirectorySearcher("(&(CN=*)(objectClass=User))")
$ds.PageSize = 900
$ds.PropertiesToLoad.addrange(('Name','Logonname','Company','division'))
$ds.CacheResults = $false
$ds.findAll() | %{$_.properties | select `
@{name='Name';expression={$_.name}},
@{name='Logonname';expression={$_.logonname}},
@{name='Company';expression={$_.company}},
@{name='division';expression={$_.Division}}}
}

it selects only the needed properties and outputs them in a nice table
(resolving propertiecollections ) you might be able to add this version to a
variable )
oMV replied on 16-Jul-07 05:10 PM
*doink* moment

ofcourse we shoud use objectCategory here as that one is indexed, (you can
add the 2 other properties you need also ofourse) will post some testresults
later

Function Get-AllAdUsers {
$ds = New-Object
System.DirectoryServices.DirectorySearcher("(&(CN=*)(objectCategory=User))")
$ds.PageSize = 900
$ds.PropertiesToLoad.addrange(('Name','Logonname','Company','division'))
$ds.CacheResults = $false
$ds.findAll() | %{$_.properties} | select `
@{name='Name';expression={$_.name}},
@{name='Logonname';expression={$_.logonname}},
@{name='Company';expression={$_.company}},
@{name='division';expression={$_.Division}}
}
(measure-command {$all = Get-AllAdUsers} ).TotalSeconds
$all.Count

Greetings /\/\o\/\/
Rostislav Pridatko [QSFT replied on 17-Jul-07 03:08 AM
You shouldn't use -IncludedAllProperties switch if you only need a couple of
user properties. Try this:
Get-QADUser -DontUseDefaultIncludedProperties -IncludedProperties
SamAccountName,proxyAddresses -PageSize 1000 -SizeLimit 0
And yes,  you could pipe the output of get-qaduser directly to
foreach-object and use the $_ predefined variable instead of $user in the
outer loop - this should save some memory, too:
Get-QADUser -DontUseDefaultIncludedProperties -IncludedProperties
SamAccountName,proxyAddresses -PageSize 1000 -SizeLimit 0 | foreach
{write-host "Working now on user " $_.SamAccountName ;$proxy =
$_.proxyAddresses
.... ,etc.
You could even convert the outer loop to a PS function to simplify its reuse.
And one more thing: you can post questions related to Quest cmdlets (*-
qad*) to the following forum:
http://www.powergui.org/forum.jspa?forumID=173
This is the support forum for the Quest AD mgmt snapin and is
monitored by the developers of this snapin (myself included).
HTH
Did replied on 19-Jul-07 12:04 AM
Hi Rostislav,
Well done ! now it is working !
Mov - Thank you too.

Didi