Yet in 2015 I wrote a post about the automatic provisioning of a PWA instance. That time, it discussed the process in the context of Project Server 2013, now I updated the description to the version 2016.

Preparations

First of all, be sure you have enabled the usage of Project Server in the farm by entering a valid license key, otherwise you receive the error message below, when enabling the PWASITE feature at the end of the provisioning:

Enable-SPFeature : The farm does not have a product key for Project Server.
You can add your key by using Enable-ProjectServerLicense

As suggested by the error message, you should add your key by using Enable-ProjectServerLicense like:

Enable-ProjectServerLicense -Key [Guid of you Project Server product key]

At this step you might receive a further error message, stating:

Enable-ProjectServerLicense : We encountered an error while enabling Project
Server 2016 . Project Server 2016  requires the farm to have a valid
SharePoint Server Enterprise product key.

It means Project Server requires a licensed Enterprise version of SharePoint. In our case it was a bit confusing, as the page Upgrade and Migration / Enable Enterprise Features in Cental Administration displayed that the farm has already the Enterprise features activated.

The Upgrade and Migration / Convert License Type answered the question, as it turned out the farm was not licensed at all, being in state:

SharePoint Server Trial with Enterprise Client Access License

After entering the correct license key for the enterprise version to the  Enter the Product Key text field an submitting the form:

On the Convert License Type page the Current License changed to:

SharePoint Server Trial with Enterprise Client Access

and the Enter the Product Key text field was no more editable.

Unfortunately, I have not found any solution to convert the farm license type by PowerShell, so if you need it, you should perform it in Cental Administration user interface manually. The process of converting a license type is discussed in this post in details.

After we have a licensed Enterprise version, we can enable the Project Server license as well. Note, that you should restart your PowerShell console after converting the license type from Cental Administration as discussed above, otherwise it seems to be unable to recognize the change, and you receive the same error message about the lack of a valid SharePoint Server Enterprise product key as earlier.

Enable-ProjectServerLicense -Key [Guid of you Project Server product key]

The result should be now:

Project Server 2016  is now enabled.

Second part of the preparation is to ensure you have the adequate permissions to perform the provisioning job. Personally I prefer to have db_owner membership in all databases in the farm, including the configuration database as well. It is particularly important to check the permissions, if you plan to use an already existing content database to provision your PWA site, that you created via Central Administration site and not by PowerShell. Based on my experience the databases created in CA have different permissions configured as the ones created by PowerShell, and the lack of the permission may cause your provisioning process to stuck with no trivial solution.

Provisioning

After being prepared, let’s see our PowerShell script that provisions a new PWA instance, including:
– A separate SharePoint content database that should contain only a single site collection: the one for the PWA. If the content DB already exists, we will use the existing one, otherwise we create a new one.
– Creating the managed path for the PWA.
– A new site collection is created for the PWA using the project web application site template, and the right locale ID (1033 in our case). If the site collection already exists (in case we re-use a former content DB), it will be dropped before creating the new one.
– The PWASITE feature will be activated on the new site collection.
– Changing to the Project Server security model.
– Disabling quotas.

A comment regarding the last step, disabling quota. If you simply read the values of the current quota using the Get-SPProjectDatabaseQuota cmdlet, disable the quota, and try to set the value by Set-SPProjectDatabaseQuota cmdlet, you get an error message:

Set-SPProjectDatabaseQuota : Cannot apply settings, the maximum database size must be greater than the read only limit.

That is because the properties MaxDbMegaByteSize and ReadOnlyMegaByteLimit have by default the same value (10240). Funny, that it later not allowed to set the same values yourself. That is why we have $quota.MaxDbMegaByteSize++ in code.

A few words about the ‘This site is read-only at the moment.‘ banner, more details about what that means can be found here and there. Recently I had the message in a new Project Server 2016 installation (so on-premise and no migration) for a PWA instance, although the site and the content database were not in read-only mode at all, as I was able to upload files into the document library of the site. I have to admit, I forgot to check, if it was possible to create Project entities, like projects or resources.

Steps I’ve performed to let the banner to disappear:

• Setting the quota multiple times
• Starting the service job Project Server: Database Maintenance job for Project Server Service Application multiple times
• Calling Enable-ProjectServerLicense multiple times
• Setting permission (db_owner) for SharePoint service users / farm account on the content database
• IISRESET

After the last step (and few hours of trying) the banner finally disappeared, although I’m not sure, if it was really an effect of the IISRESET, or some kind of combination of the other steps played a role in the solution as well. I thing the permission change might be important as well, although after revoking them and running IISRESET once more the banner did not come back again.

While supporting a SharePoint environment having several thousands of users from multiple Active Directory domains, we have quite often complains, that one can not access the site. Beyond trivial network related problems, like incorrect proxy settings, it is probably the second most common reason for such complains having issues with the Active Directory account of the user.

To support such cases, I wrote a short PowerShell script that checks for the most common problems, like User must change password at next logon flag is activated, account is disabled or locked out, and password expired. Prerequisite: you should have PowerShell Active Directory module installed.

$userLogin = ‘Domain\UserName’
$userLoginSplitted = $userLogin.Split(‘\’)
$domainName = $userLoginSplitted[0]
$dcServer = Get-ADDomainController -Discover -DomainName $domainName

$user = Get-ADUser -Identity $userLoginSplitted[1] -Server $dcServer.HostName[0] -Properties Enabled, LockedOut, PwdLastSet, PasswordNeverExpires, msDS-UserPasswordExpiryTimeComputed
$pwdNeverExp = $user.PasswordNeverExpires
$pwdExpiresOn = If ($pwdNeverExp) { $null } Else { [DateTime]::FromFileTime($user."msDS-UserPasswordExpiryTimeComputed") }

Write-Host Checking user: $userLogin
Write-Host User must change password at next logon: $($user.PwdLastSet -eq 0)
Write-Host Account disabled: $(!$user.Enabled)
Write-Host Account locked out: $($user.LockedOut)
Write-Host Password expired: $((!$pwdNeverExp) -and ($pwdExpiresOn -lt [DateTime]::Now))

First thing to highlight in the script is how we get and use the domain controller. Getting the domain controller is the easy part, after splitting the user login name to a domain name and a user name, you simply invoke the Get-ADDomainController cmdlet with the Discover switch and passing the domain name you are looking for in the DomainName parameter.

Using the value returned is a bit more complicated, at least until you learn how to do it the right way. Although Active Directory cmdlets support the Server parameter, there is a lot of confusion how to use it correctly. If you simply pass the domain controller as you received it in the previous step from the Get-ADDomainController cmdlet, like:

$user = Get-ADUser -Identity $userLoginSplitted[1] -Server $dcServer

you receive this error message:

Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running.

After researching the samples on the web (like Get-ADUser -Server ‘servername’) and the official documentation of the Get-ADUser cmdlet we realized that the Server parameter requires a string value, so passing the $dcServer (of type Microsoft.ActiveDirectory.Management.ADDirectoryServer) was really not a good idea. But wait, scrolling through the properties of $dcServer by PowerShell autocomplete shows it has a property called HostName. That sounds really promising! Try it out!

$user = Get-ADUser -Identity $userLoginSplitted[1] -Server $dcServer.HostName

Now you have another error message:

Cannot convert ‘Microsoft.ActiveDirectory.Management.ADPropertyValueCollection’ to the type ‘System.String’ required by parameter ‘Server’. Specified method is not supported.

Well, that means that the HostName property is of type Microsoft.ActiveDirectory.Management.ADPropertyValueCollection and not a string either, even though PowerShell displays it as string if you output it like $dcServer.HostName. To get the name of the domain controller as string you should address it in the collection using an array indexer, like $dcServer.HostName[0].

So the right usage is:

$user = Get-ADUser -Identity $userLoginSplitted[1] -Server $dcServer.HostName[0]

although you might check first if there was any domain controller found ($dcServer and its HostName property not null, and HostName has at leas a single entry) if you want to be sure.

Note furthermore, that the largest possible 64-bit value (2^63-1 = 9223372036854775807) in property msDS-UserPasswordExpiryTimeComputed means PasswordNeverExpires is true. Invoking the FromFileTime method using this value would cause an error:

Exception calling "FromFileTime" with "1" argument(s): "Not a valid Win32 FileTime. Parameter name: fileTime"

Today I received an Excel sheet that I should have to edit, although some cells whose value I had to change was protected by a password I did not know (Note: I’m using Excel 2013).

I’m quite familiar with Open XML, so I was sure, it must be possible by editing the XML files the Excel file consist of, but did not know exactly, which nodes should I edit or remove. A quick search provided me the information, how to do it manually or by using the Open XML SDK and C#.

Although I like C# and have Visual Studio installed on my PC, I decided to create a solution using PowerShell based on the C# example mentioned above, to enable the automated removal of password protection for a broader range of users. Prerequisite: the Open XML SDK must be installed on the computer.

The conversion was not really complex, a key point was the usage of the MakeGenericMethod to enable access to the generic methods of the Open XML object model.

The code snippet below illustrates the result:

As mentioned in the comment in the code, you should change the path to refer to the real location of your Excel file, and mustn’t have the document opened in Excel, otherwise it will lock the file and the script can’t change it. If you want to be sure, close all instances of Excel before staring the script.

Recently I found a document library in one of our SharePoint 2013 environments, that produced an odd behavior. I describe the symptoms and the solution below.

Symptoms

Problem with uploading file into a document library

I opened the Site Assets document library in a web site of a SharePoint 2013 installation (I don’t think the error is specific to this special kind of document libraries or SharePoint version, but I document it for the sake of completeness), and created a folder called ‘js’. The folder was created without any problem. I wanted to copy a .js file into the folder, but received the error message “The file name you specified is not valid or too long”. There was of course no problem with the length of the file nor with the name itself. I was simply not able to upload any file into that specific library, independent of the folder or the file name and extension. See below an error message for a text file. Clicking “Try Again” was unsuccessful.

I was able to upload files into other document libraries of the same site, as well as into libraries in other sites, so I assumed the problem is related to the library itself, but to be sure, I restarted first the WebClient service in Windows, then the whole Explorer process (including the desktop), finally the operating system itself, but all of them without any result (except losing lot of time).

I observed, however, that the file was uploaded despite of the error message. It appeared both in Windows Explorer as well as in the browser (I was able to open it!), at least till I clicked “Cancel” in the dialog box above. Then the file got deleted and disappeared, as it would have not been uploaded at all. Very strange.

I created a network trace using Fiddler to investigate the steps of the process. I include the HTTP response codes and optionally further information in brackets, unsuccessful steps are written in red:

PROPFIND /YourSite/SiteAssets (207)
PROPFIND / (207)
PROPFIND /YourSite/SiteAssets/YourFile.txt (404)
PROPFIND /YourSite/SiteAssets (207)
PUT /YourSite/SiteAssets/YourFile.txt (In the request we send the X-MSDAVEXTLockTimeout header with a value of Second-3600 and do not include the file in the request body, it indicates we acquire only a lock. The response includes these headers: X-MSDAVEXTLockTimeout: Second-3600; Lock-Token: opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z; X-MS-File-Checked-Out: 1. The HTTP response code is 201)
HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt We include the lock token we acquired earlier: Lock-Token: <opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>, another header X-MSDAVEXT: PROPPATCH and the file to be uploaded in the request body. The HTTP response code is 500, the response includes the header X-MSDAVEXT_Error: 589829; The%20URL%20%27SiteAssets%2fYourFile%2etxt%27%20is%20invalid%2e%20%20It%20may%20refer%20to%20a%20nonexistent%20file%20or%20folder%2c%20or%20refer%20to%20a%20valid%20file%20or%20folder%20that%20is%20not%20in%20the%20current%20Web%2e)
HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt (It’s not clear what happens in this step, we try to upload the file again, but with another headers this time, like If: (<opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>) and there is no X-MSDAVEXT. The file to be uploaded is included again in the request body. The HTTP response code is 500, the response includes the header X-MSDAVEXT_Error value as earlier.)
PROPFIND /YourSite/SiteAssets (207)
PROPFIND /YourSite/SiteAssets/desktop.ini HTTP/1.1 (404)

After clicking “Cancel”:

HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt HTTP/1.1 (This time we send only a DAV property update request, the file is not included in the request body. The HTTP response code is 500, the response includes the header X-MSDAVEXT_Error value as earlier.)
HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt HTTP/1.1 (Another put request with the header If: (<opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>) but without the file in the request body. The HTTP response code is again 500, the response includes the header X-MSDAVEXT_Error value as earlier.)
UNLOCK http://YourServer/YourSite/SiteAssets/MailAdresses.txt (we try to unlock the file, including the header Lock-Token: <opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>. The HTTP response code is 412)
PROPFIND /YourSite/SiteAssets/YourFile.txt (207)
DELETE http://YourServer/YourSite/SiteAssets/YourFile.txt (We delete the file. The HTTP response code is 204)

In the corresponding ULS log entries I found these lines (all entry with Area: SharePoint Foundation, Category: Database; Level: High):

System.Data.SqlClient.SqlException (0x80131904): Parameter ‘@tp_Version’ was supplied multiple times. at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows) at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more) at System.Data.SqlClient.SqlDataRea… …der.TryNextResult(Boolean& more) at System.Data.SqlClient.SqlDataReader.NextResult() at Microsoft.SharePoint.SPSqlClient.ExecuteQueryInternal(Boolean retryfordeadlock) at Microsoft.SharePoint.SPSqlClient.ExecuteQuery(Boolean retryfordeadlock) ClientConnectionId:0ed39f0d-b333-43fd-90e3-421a19b57441 Error Number:8143,State:1,Class:16 ExecuteQuery failed with original error 0x80131904 SQL error code from last error 8143 – Parameter ‘@tp_Version’ was supplied multiple times.

Creating new files into a document library

Recently we detected that a lot of deleted calendar entries appeared in the Recycle Bin of a SharePoint site. They were all originated from a specific calendar instance, and deleted by a custom application (written in C#) that is scheduled to run regularly to purge old entries.

Although we could have deleted the items using the bulk method described here, in this case we deleted the entries one by one. To get IDs of the “master” entries (entries that are not recurring event exception) we should delete, we run a CAML query like the below one, filtering on the EventType field of the items, just as described in my former post:

<Where>
    <And>
    <Lt>
        <FieldRef Name=’EndDate’ />
        <Value Type=’DateTime’>2018-01-01 00:00:00</Value>
    </Lt>
    <Lt>
        <FieldRef Name=’EventType’/>
        <Value Type=’Integer’>2</Value>
    </Lt>
    </And>
</Where>

You can read more about the meaning of the various EventType values in this blog post.

As I wrote in my post, we should (at least, theoretically, see explanation later) delete only the main entries, as all of the related entries (the recurring event exceptions, deleted and changed instances of the series) are deleted automatically by the system when you delete the main entry.

Having the IDs from the CAML query, we deleted the entries by iterating through the collection of IDs and invoked the the DeleteItemById method of the SPListItemCollection method, like:

foreach (int itemIDToDelete in itemIDsToDelete)
{
    calendarList.Items.DeleteItemById(itemIDToDelete);
}

As you probably know, you can delete an item from code by invoking the Delete method of the SPListItem instance of the item (in this case the item is deleted immediately, without being recycled), or recycle it by calling the Recycle method of the SPListItem instance (in this case the item is simply moved to the Recycle Bin and you can restore it later if you wish). I should point out, that both of these methods call internally the private DeleteCore method of the SPListItem class, using the parameter value DeleteOp.Delete in the first case and DeleteOp.Recycle in the second case.

The DeleteItemById method invokes the Delete method however, so it should not miss up the Recycle Bin, as it obviously did in our case.

public void DeleteItemById(int id)
{
    this.GetItemById(id).Delete();
}

So what was the problem? After a short investigation and running a few tests, we found, that only recurring event exceptions got moved to the Recycle Bin when the system delete them automatically, the main entries were deleted permanently. It means, that the Delete method of the SPListItem class is buggy and the same is true for the DeleteItemById method, at least I don’t consider this behavior to be some kind of hidden feature.

How to solve the problem? If you have a lot of recurring event exceptions in your calendar, and don`t want to pollute your Recycle Bin, the best you can do to create some kind of extension method that deletes the related entries (recurring event exceptions) explicitly, not letting the system to delete them automatically.

I’ve created two extension methods, as shown below:

You can use the DeleteItemByIdIncludingRecurringEventExceptions method in place of the DeleteItemById method and DeleteIncludingRecurringEventExceptions method in place of the Delete method. We search for the related items by using the MasterSeriesItemID field value in the CAML query. You can use these methods only for items having the Event content type or a custom content type derived from it.

You can use the methods like this:

var list = web.Lists["Calendar"];          
list.Items.DeleteItemByIdIncludingRecurringEventExceptions(10);

Note, that we also had some corrupted recurring event exceptions in our calendar, probably created automatically by a faulty application. Although we could get a reference for the items itself (for example, by calling the GetItemById method), and change its properties if we wished, we got the exception below if we tried to delete them from code, or even if we only wanted to display the items from the All Events view in the browser.

Item does not exist. The page you selected contains an item that does not exist. It may have been deleted by another user.<nativehr>0x81020016</nativehr><nativestack></nativestack>
SPRequest.DeleteItem: UserPrincipalName=i:0).w|s-1-5-21-3634847118-2359816030-2114994487-3414, AppPrincipalName= ,bstrUrl=http://YourServer/Web/SubWeb ,bstrListName={A38F8D71-F481-4A93-85B8-AC42BB2BE6EC} ,lID=4596 ,dwDeleteOp=3 ,bUnRestrictedUpdateInProgress=False
System.Runtime.InteropServices.COMException: Item does not exist. The page you selected contains an item that does not exist. It may have been deleted by another user.<nativehr>0x81020016</nativehr><nativestack></nativestack>, StackTrace:    at Microsoft.SharePoint.SPListItem.DeleteCore(DeleteOp deleteOp)     at Microsoft.SharePoint.SPListItem.Delete()     at CallSite.Target(Closure , CallSite , Object , Int32 )     at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)     at System.Management.Automation.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.Ente…
…rTryCatchFinallyInstruction.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0)     at System.Management.Automation.DlrScriptCommandProcessor.RunClause(Action`1 clause, Object dollarUnderbar, Object inputToProcess)     at System.Management.Automation.DlrScriptCommandProcessor.Complete()     at System.Management.Automation.CommandProcessorBase.DoComplete()     at System.Management.Automation.Internal.PipelineProcessor.DoCompleteCore(CommandProcessorBase commandRequestingUpstreamCommandsToStop)     at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input, Hashtable errorResults, Boolean enumerate)     at S…
…ystem.Management.Automation.Runspaces.LocalPipeline.InvokeHelper()     at System.Management.Automation.Runspaces.LocalPipeline.InvokeThreadProc()     at System.Management.Automation.Runspaces.PipelineThread.WorkerProc()     at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)     at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)     at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)     at System.Threading.ThreadHelper.ThreadStart() 

The simplest workaround I’ve found to delete those entries was to set their EventType field value to 0, as they would be master entries, not recurring event exceptions. After this change, it was possible to delete the items. This kind of hack is also included in the DeleteIncludingRecurringEventExceptions extension method above.

Although the code I provided here seems to do the job and not pollute the Recycle Bin any more, if you have a really large number of items to delete, for performance reasons I still would prefer the bulk deletion of the events, or you should write your own solution to select all of the recurring event exceptions in the first step, and deleting them before you delete the main entries in the second step. I don’t think it would be a great idea to run a separate CAML query for each recurring events in your calendar.