.NET Framework - [GC]::COLLECT() Doesn't work

Asked By RickB
03-Jul-08 07:08 AM
I'm doing queries against an SQL Server 2005 database.
After calling ...Fill($records) and processing $records the routine
exits.
An outer loop calls it again.  Pretty soon I get an out-of-memory
exception.
I tried putting [gc]::collect() in the routine that creates $records
but it doesn't help.
Then I tried from the command prompt.

PS 20> [GC]::GETTOTALMEMORY($TRUE)
1379511036
PS 21> [GC]::GETTOTALMEMORY($TRUE)
1379510280
PS 22> [GC]::COLLECT()
PS 23> [GC]::GETTOTALMEMORY($TRUE)
1379510188

None of the variables I use in any of this processing exist any more
(that I can tell).
Any suggestions?


In case the devil is in the details, this is all the code involved in
the process.


Get-Table_Names emo___|%{Generate-PLan_Snapshot $_.table $_.schema 93 -
m}

function Get-Table_Names ([string]$template = '%'
,[string]$schema = '%'
,[string]$database = 'Raser_Rpt'
,[switch]$Q,[switch]$M) {
(Execute-SQLcmd "select RS.[NAME] as [schema], RT.[name] as
[table]
from sys.tables as Rt
join sys.schemas as Rs on Rt.schema_id = Rs.schema_id
where Rs.[name] like '$schema'    -- schema name pattern
and Rt.[name] like '$template'  -- table  name pattern


function Generate-Plan_Snapshot ([string]$Table,[string]$Schema =
'dbo',
[string]$Plan,[int]$Timeout=300,[switch]$View,[switch]
$Measure){
$ErrorActionPreference = 'stop'
[GC]::Collect()
$RunDate = get-date
$Abbr    = (Execute-SQLcmd "select Abbreviation from R_D.Company_Codes
where Company_Code = '$Plan'" `
-M:
$Measure).tables.item(0).rows.item(0).item(0)
$File    = ("Q:\Temp\Full_Snapshot\{0,2}pdclms{1,6}.
{2:yyyyMMdd.HHmmss}.txt" -f $Abbr,$table,$RunDate).ToLower()
$Cutoff  = (Execute-SQLcmd "select max(process_date) from $Schema.
$Table" -t $timeout `
-M:
$Measure).tables.item(0).rows.item(0).item(0)
$Records = Execute-SQLcmd "select * from Snapshot_$Table('$Plan')" -t
$timeout -M:$Measure
if ($Records.tables.item(0).rows.count -eq 0){throw "$table has no
records for $plan"}
$RecLen  = $Records.tables.item(0).rows.item(0).item(0).length
function i_generate(){
($RecLen -45)}" -f
$RunDate,$Cutoff,$Records.tables.item(0).rows.count,''
$records.tables.item(0)|%{$_.item(0)}
($RecLen -45)}" -f
$RunDate,$Cutoff,$Records.tables.item(0).rows.count,''}
if ($view){i_generate|write-host}else{i_generate|out-file $File -
encoding ASCII}}

command(s)
function Execute-SQLcmd ([string[]]$cmd
,[int]$timeout=120
,[string]$database = 'Raser_Rpt'
,[string]$server=$(if ($server -like
'*dras01')

{"$server"}else{"pmdras01"})
,[switch]$M
,[switch]$Q,
[switch]$debug)
{   Write-Debug "$cmd" -debug:$debug
if ($M){write-host ('{0:T}' -f $(get-date))}
if ($cmd.Count -ne $null -and $cmd.Count -ne 0)
{   $SqlAdapter     = New-Object
System.Data.SqlClient.SqlDataAdapter
$SqlConnection  = New-Object
System.Data.SqlClient.SqlConnection
$SqlCmd         = New-Object System.Data.SqlClient.SqlCommand
$DataSet        = New-Object System.Data.DataSet
$SqlConnection.ConnectionString = "Server=$server;Database=
$database" +
Security=True"
$SqlCmd.Connection = $SqlConnection
$SqlCmd.CommandTimeout = $timeout
$SqlAdapter.SelectCommand = $SqlCmd
$SqlCmd.CommandText = $cmd
if ($Q) {[void]$SqlAdapter.Fill($DataSet)}
Else {Write-Host "Rows Affected = $
($SqlAdapter.Fill($DataSet))"}
if ($M){write-host ('{0:T}' -f $(get-date))}
$DataSet
$SqlConnection.Close()}}
System.Data.SqlClient.SqlDataAdapter
(1)
System.Data.SqlClient.SqlCommand
(1)
SQL Server 2005
(1)
System.Data.DataSet
(1)
Windows Server
(1)
SqlAdapter.SelectCommand
(1)
SqlCmd.CommandTimeout
(1)
SqlCmd.CommandText
(1)
  Tao Ma replied...
02-Jul-08 10:11 AM
Hi RickB,

I think you'd better call Clear() method on the Dataset object once it
becomes unused. GC::Collect don't guarantee reclaiming memory immediately.

Function Generate-Plan_Snapshot calls Execute-SQLcmd 3 times, I think you
can release them before you quit. I hope this is the error-prone.

Best regards,
Tao Ma

??????:8d9a733d-6186-44d5-93a8-a18fbb97bcd1@i76g2000hsf.googlegroups.com...
  Marco Shaw [MVP] replied...
02-Jul-08 10:22 AM
Also actually removing the variables manually may also be required to be
able to call the GC just after or whenever you want.

Marco

--
*Microsoft MVP - Windows Server - Admin Frameworks
https://mvp.support.microsoft.com/profile/Marco.Shaw
*PowerShell Co-Community Director - http://www.powershellcommunity.org
*Blog - http://marcoshaw.blogspot.com
  RickB replied...
03-Jul-08 07:08 AM
On Jul 2, 9:22=A0am, "Marco Shaw [MVP]" <marco.shaw@_NO_SPAM_gmail.com>
y.
osoft.com/profile/Marco.Shaw

I added $Records.clear() to the end of Generate-Plan_Snapshot but it
didn't
make any difference.  Basically either PowerShell dies or I need to
restart it
between queries.  As my initial post suggests, I don't see anything
happening
after using GetTotalMemory($true) or Collect() and the variables
should be
out of scope (and therefore collectable).

If you're curious I have successfully created a 674,512KB file with
these
functions.  I'd actually like to find a way to create one thats close
to 1GB.
The machine has enough memory for it but I can't tell where the
application
limits are.

Are there memory limits on each object or just total memory
allocated?
Does anyone know what they are?
  RickB replied...
03-Jul-08 07:08 AM
ely.

crosoft.com/profile/Marco.Shaw

It seems I spoke too soon.
I actually AM seeing memory being released using the combination
of clear() and collect.

Thanks!

I would still like to know something about the limits if anyone
knows (or knows where to look).
Create New Account
help
Sql scripts .NET Framework Just looking for examples of what people have been doing using PS and Sql2005 Backups Maint Thanks. Powershell Discussions Microsoft.SqlServer.Management.Smo.Server (1) System.Data.SqlClient.SqlDataAdapter (1) System.Data.SqlClient.SqlCommand (1) SQL Server 2005 (1) Microsoft.SqlServer.ConnectionInfo (1) System.Data.DataTable (1
help please .NET Framework I' m going slightly mad with powershell and SQL express 2005 but i love it My problem is I have a text file with some information site statistics with microsoft chart But i m stopped here $c = Get-Credential Connect-VIServer -server nameserver -Credential $c Get-Stat -Entity (Get-vmhost -Name "servername*") -stat mem.usage.average -maxsamples francesco \ stat.txt $SqlConnection = New-Object System.Data.SqlClient.SqlConnection $SqlConnection.ConnectionString = $SqlCmd = New-Object System.Data.SqlClient.SqlCommand $SqlCmd.CommandText = get-content c: \ francesco \ stat.txt $sqlDataAdapter = new-object System.Data.SqlClient.SQLDataAdapter($sqlcommand) $sqlDataSet = new-object System.Data.dataset $SqlAdapter.Fill($DataSet) I m
How to write records to SQL Server 2005 tables .NET Framework I have a PowerShell routine that collects data. Now I need to write several dozen records based on this data to a table in an SQL Server 2005 database. I've written queries that return datasets but I'm not sure how to go the other direction. Thanks Powershell Discussions SQL Server (1) Windows Server (1) PowerShell (1) SqlConnection (1) SqlCmd (1) SqlAdapter (1) SqlClient (1) MVP