About a year ago I wrote about how you can use PowerShell and Project Server REST interfaces (ProjectServer and ProjectData) to generate reports. However, at the end of last year we had a problem with the script that query the projects via the ProjectData interface:
- $url = 'http://YourProjectServer/PWA/_api/ProjectData/%5Ben-US%5D/Projects?$select=ProjectId,ProjectName,ProjectCreatedDate'
- $reportPath = "C:\Data\ProjServerReports\"
- $request = [System.Net.WebRequest]::Create($url)
- $request.UseDefaultCredentials = $true
- $request.Accept = "application/json;odata=verbose"
- $response = $request.GetResponse()
- $reader = New-Object System.IO.StreamReader $response.GetResponseStream()
- $data = $reader.ReadToEnd()
- $result = ConvertFrom-Json -InputObject $data
- $result.d.results | select ProjectId, ProjectName, ProjectCreatedDate | Export-Csv -Path ($reportPath + 'ProjectsFromProjectDataNoBatching.csv') -Delimiter ";" -Encoding UTF8 -NoTypeInformation
Important! You should use single quotes around the URL in this code snippet and in the other ones below to avoid PowerShell to remove the string $select in the REST query. More about that here.
A user reported an error that some of the projects were not included in the results. We found that the count of projects included in the report via ProjectServer was greater than the count of the projects we queried via ProjectData. It is the script version that use the ProjectServer interface:
- $url = 'http://YourProjectServer/PWA/_api/ProjectServer/Projects?$select=Id,Name,CreatedDate'
- $reportPath = "C:\Data\ProjServerReports\"
- $request = [System.Net.WebRequest]::Create($url)
- $request.UseDefaultCredentials = $true
- $request.Accept = "application/json;odata=verbose"
- $response = $request.GetResponse()
- $reader = New-Object System.IO.StreamReader $response.GetResponseStream()
- $data = $reader.ReadToEnd()
- $result = ConvertFrom-Json -InputObject $data
- $result.d.results | select Id, Name, CreatedDate | Export-Csv -Path ($reportPath + 'ProjectsFromProjectServer.csv') -Delimiter ";" -Encoding UTF8 -NoTypeInformation
First I thought it is an issue with some permissions, but it was suspicious, that all of the missing projects were at the end of the project list (when ordered alphabetically), and the number of included project was exactly 100 in the ProjectData-based report.
Then I found this statement in the ProjectData – Project OData service reference:
“There are limits to the number of entities that can be returned in one query of the ProjectData service.”
What a surprise, the default limit for projects in the on-premise version we have is exactly 100!
You can use the Set-SPProjectOdataConfiguration PowerShell cmdlet for on-premise instances of Project Server to override the default query page size limits for any specified entity set.
Set-SPProjectOdataConfiguration -EntitySetName Projects -PageSizeOverride 200
However, it is a global change on the server, that affects all queries of the specific entity type, that can adversely the server performance, and as such, is not recommended. Instead of this, the suggested solution to query all instance of a given entity type is to implement paging by sending multiple queries using the $top and $skip operator.
Instead of a fix URL we define only a formatting pattern with placeholders for the $skip and $top parameters for the subsequent queries. More about this format is available here.
By applying the $inlinecount=allpages in the query we can assure that the result is always returned in $result.d.results as described in this post.
In the script we assume that the default limit of 100 project / query has not been changed. Note, that we use the $firstBatch variable to decide, if we should append the data to the existing report file in the Export-Csv cmdlet, or to create a new one.
In the first version of the script we process the results (output them into a CSV) immediately after receiving the response.
- $urlPattern = 'http://YourProjectServer/PWA/_api/ProjectData/%5Ben-US%5D/Projects?$select=ProjectId,ProjectName,ProjectCreatedDate&$skip={0}&$top={1}&$inlinecount=allpages'
- $reportPath = "C:\Data\ProjServerReports\"
- $firstItem = 0
- $batchSize = 100
- $firstBatch = $true
- Do
- {
- Write-Host Requesting next batch of $batchSize items, starting at $firstItem
- $url = $urlPattern -f $firstItem, $batchSize
- Write-Host $url
- $request = [System.Net.WebRequest]::Create($url)
- $request.UseDefaultCredentials = $true
- $request.Accept = "application/json;odata=verbose"
- $response = $request.GetResponse()
- $reader = New-Object System.IO.StreamReader $response.GetResponseStream()
- $data = $reader.ReadToEnd()
- $result = ConvertFrom-Json -InputObject $data
- $count = $result.d.results.Count
- Write-Host Item count is $count
- $result.d.results | select ProjectId, ProjectName, ProjectCreatedDate |
- Export-Csv -Path ($reportPath + 'ProjectsFromProjectData_20171130_03.csv') -Delimiter ";" -Encoding UTF8 -NoTypeInformation -Append:(-not $firstBatch)
- $firstBatch = $false
- $firstItem += $batchSize
- }
- While ($count -eq $batchSize)
In the second we aggregate the results into an array, and process the results only after receiving the last response.
- $urlPattern = 'http://YourProjectServer/PWA/_api/ProjectData/%5Ben-US%5D/Projects?$select=ProjectId,ProjectName,ProjectCreatedDate&$skip={0}&$top={1}&$inlinecount=allpages'
- $reportPath = "C:\Data\ProjServerReports\"
- $results = @()
- $firstItem = 0
- $batchSize = 100
- Do
- {
- Write-Host Requesting next batch of $batchSize items, starting at $firstItem
- $url = $urlPattern -f $firstItem, $batchSize
- Write-Host $url
- $request = [System.Net.WebRequest]::Create($url)
- $request.UseDefaultCredentials = $true
- $request.Accept = "application/json;odata=verbose"
- $response = $request.GetResponse()
- $reader = New-Object System.IO.StreamReader $response.GetResponseStream()
- $data = $reader.ReadToEnd()
- $result = ConvertFrom-Json -InputObject $data
- $count = $result.d.results.Count
- Write-Host Item count is $count
- $results += $result.d.results
- $firstItem += $batchSize
- }
- While ($count -eq $batchSize)
- $results | select ProjectId, ProjectName, ProjectCreatedDate |
- Export-Csv -Path ($reportPath + 'ProjectsFromProjectDataWithBatching.csv') -Delimiter ";" -Encoding UTF8 -NoTypeInformation
Although in this case both of the scripts have the same result, you can use the second version if you need the complete list of the projects at a later time, for example, you would like to extend it with data not available at the time of the query.