Wallace Turner Wallace Turner programming articles https://wallaceturner.azurewebsites.net/feed 2019-09-11T01:13:21Z https://wallaceturner.azurewebsites.net/a-free-highly-availabile-database A free Highly Availabile Database <p>The code for this post is <a href="https://github.com/wallaceturner/ravendb-vagrant">available on GitHub</a></p> <p>With the release of RavenDb4 and in combination with their <a href="https://ravendb.net/license/request/community">Community License</a> and Ubuntu Server Edition it is now possible to deploy a highly available database setup that is free of software licensing costs (server hosting costs obviously still apply)</p> <div style="background: #FFEEF2 -webkit-linear-gradient(#FFEEF2, #fdbebe); padding: 5px; border: 1px solid black;"> The source code provides a server.pfx and client.pfx which are obviously for TEST environments only. Do not use these when rolling out to production. There i warned you. In fact you may want to consider the RavenDb Wizard setup and/or RavenDb Cloud offering if you are looking for a more automated setup </div> <p>To demonstrate a possible setup clone my <a href="https://github.com/wallaceturner/ravendb-vagrant">vagrant-ravendb</a>, navigate to the root folder and run</p> <pre><code>vagrant up </code></pre> <p>This will reproduce the network shown below: (<em>as an aside, if you haven't already checked out <a href="https://www.vagrantup.com/">vagrant</a> i highly recommend doing so</em>)</p> <p><img src="https://wallaceturner.azurewebsites.net/get/network_raven_ubuntu.png" alt="Network Overview" title="Network Overview" /></p> <h3>Cluster Setup</h3> <p>The nice part is that you can perform the setup from a single instance. </p> <p>As the certificates are self-signed you will need to trust the certificate in your host by running</p> <pre><code>certutil -importpfx -f -user ./install_files/client.pfx </code></pre> <p>Navigate to <code>https://raven1.mooo.com:8080</code> and enter your license by clicking the 'License Information' in the toolbar at the bottom on the right hand side. (do NOT enter the license on the other nodes as it will prevent them from being added to the cluster)</p> <p>Navigate to <code>Manage Server -&gt; Cluster</code> and click <code>Add Node To Cluster</code> - in the <code>URL</code> field enter the names of the other nodes e.g. <code>https://raven1.mooo.com:8080/</code></p> <p>Your cluster is now setup and should look like</p> <p><img src="https://wallaceturner.azurewebsites.net/get/cluster_topology.png" alt="Cluster" /></p> <p>From here you can create a new database <code>Databases -&gt; New database</code> named <code>TestDb</code> and use the provided nodejs client application to connect to your cluster. </p> <p>Navigate to the <code>/client</code> folder in the source code and execute</p> <pre><code>npm i tsc node index.js </code></pre> <p>This app will insert a document into the cluster every 5 seconds:</p> <pre><code>saved new message with id messages/1-B saved new message with id messages/2-B </code></pre> <p>In this example you can see the client has connected to Node B (raven2) thereafter the document is propagated to the other nodes in the cluster. You can use this to experiment with the cluster and see how it recovers after you turn the nodes off and on.</p> <p>To see the specifics of how to setup RavenDb on Ubuntu see <code>install.sh</code> in the root folder. </p> <p>I think its incredibly impressive that you can leverage this product to give a free HA database. Let us know your thoughts in the comments.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/a-free-highly-availabile-database" /> 2019-09-10T00:01:00Z 2019-09-11T01:13:21Z Wallace Turner <p>The code for this post is <a href="https://github.com/wallaceturner/ravendb-vagrant">available on GitHub</a></p> <p>With the release of RavenDb4 and in combination with their <a href="https://ravendb.net/license/request/community">Community License</a> and Ubuntu Server Edition it is now possible to deploy a highly available database setup that is free of software licensing costs (server hosting costs obviously still apply)</p> <div style="background: #FFEEF2 -webkit-linear-gradient(#FFEEF2, #fdbebe); padding: 5px; border: 1px solid black;"> The source code provides a server.pfx and client.pfx which are obviously for TEST environments only. Do not use these when rolling out to production. There i warned you. In fact you may want to consider the RavenDb Wizard setup and/or RavenDb Cloud offering if you are looking for a more automated setup </div> <p>To demonstrate a possible setup clone my <a href="https://github.com/wallaceturner/ravendb-vagrant">vagrant-ravendb</a>, navigate to the root folder and run</p> <pre><code>vagrant up </code></pre> <p>This will reproduce the network shown below: (<em>as an aside, if you haven't already checked out <a href="https://www.vagrantup.com/">vagrant</a> i highly recommend doing so</em>)</p> <p><img src="https://wallaceturner.azurewebsites.net/get/network_raven_ubuntu.png" alt="Network Overview" title="Network Overview" /></p> <h3>Cluster Setup</h3> <p>The nice part is that you can perform the setup from a single instance. </p> <p>As the certificates are self-signed you will need to trust the certificate in your host by running</p> <pre><code>certutil -importpfx -f -user ./install_files/client.pfx </code></pre> <p>Navigate to <code>https://raven1.mooo.com:8080</code> and enter your license by clicking the 'License Information' in the toolbar at the bottom on the right hand side. (do NOT enter the license on the other nodes as it will prevent them from being added to the cluster)</p> <p>Navigate to <code>Manage Server -&gt; Cluster</code> and click <code>Add Node To Cluster</code> - in the <code>URL</code> field enter the names of the other nodes e.g. <code>https://raven1.mooo.com:8080/</code></p> <p>Your cluster is now setup and should look like</p> <p><img src="https://wallaceturner.azurewebsites.net/get/cluster_topology.png" alt="Cluster" /></p> <p>From here you can create a new database <code>Databases -&gt; New database</code> named <code>TestDb</code> and use the provided nodejs client application to connect to your cluster. </p> <p>Navigate to the <code>/client</code> folder in the source code and execute</p> <pre><code>npm i tsc node index.js </code></pre> <p>This app will insert a document into the cluster every 5 seconds:</p> <pre><code>saved new message with id messages/1-B saved new message with id messages/2-B </code></pre> <p>In this example you can see the client has connected to Node B (raven2) thereafter the document is propagated to the other nodes in the cluster. You can use this to experiment with the cluster and see how it recovers after you turn the nodes off and on.</p> <p>To see the specifics of how to setup RavenDb on Ubuntu see <code>install.sh</code> in the root folder. </p> <p>I think its incredibly impressive that you can leverage this product to give a free HA database. Let us know your thoughts in the comments.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/a-free-highly-availabile-database" /> https://wallaceturner.azurewebsites.net/ravendb-4-part-2-client-configuration RavenDb 4 Part 2 - Client Configuration <p>Part 1 of this article can be found <a href="https://wallaceturner.azurewebsites.net/ravendb-4-tryout-part-1-installation">here</a></p> <p>View the <a href="https://www.youtube.com/watch?v=Yg_XAF6NJgM">youtube</a> walkthrough of this post which contains a few extra points of interest.</p> <p>Architecture Diagram</p> <p><img src="https://wallaceturner.azurewebsites.net/get/raven-network.png" alt="mraven-network.png" /></p> <p>Now we have our RavenDb servers set up lets create a web app from scratch that will communicate with the RavenDb cluster.</p> <pre><code>dotnet new webapp -o ravendemo dotnet add package RavenDB.Client --version 4.1.5 dotnet add package Microsoft.AspNetCore.Hosting.WindowsServices </code></pre> <p>Lets now add some code to configure the RavenDb <code>DocumentStore</code> singleton. Be sure to copy <code>client.pfx</code> to the output directory. For brevity the definition of the `Order class has been omitted.</p> <pre><code>var store = new DocumentStore { Urls = new[] { "https://raven1.mooo.com:8080" }, Database="TestDb"}; store.Conventions.IdentityPartsSeparator = "-"; X509Certificate2 certificate = new X509Certificate2(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "client.pfx")); store.Certificate = certificate; store.Initialize(); using(var session = store.OpenSession()) { session.Store(new Order { Name=Guid.NewGuid().ToString()}); session.SaveChanges(); } </code></pre> <p>When you run this you receive an error</p> <blockquote> <p>The SSL connection could not be established, see inner exception. -> The remote certificate is invalid according to the validation procedure.'</p> </blockquote> <p>The certificate is not trusted as it is signed by an untrusted CA. Fix this by importing client.pfx which also contains the CA certificate. Note that this time we will import the certificate into <code>Local Computer</code> by omitting the <code>-user</code> flag. You will thus need to run the following in an Admin Powershell prompt:</p> <pre><code>certutil -f -importpfx z:\client.pfx </code></pre> <p>Re-run the app in Visual Studio Code and it should work this time. This approach has a drawback in that <code>client.pfx</code> must be used and the password exposed potentially in a config file. An alternative is to use the Windows Certificate Store to grant access to the private key. Remove the code loading the client.pfx from a file and replace with:</p> <pre><code>var x509 = new X509Store(StoreLocation.LocalMachine); x509.Open(OpenFlags.ReadOnly); var certificate = x509.Certificates.Find(X509FindType.FindBySubjectName, "*.mooo.com", true)[0]; </code></pre> <p>If you re-run the app it will <em>most</em> likely work. This is due to the fact that by default the Administrators will have access to the private keys of the cert that was imported prior. If you are running in a more locked down environment (or clear out the users in the dialog shown below) you get this error:</p> <blockquote> <p>System.ComponentModel.Win32Exception: The credentials supplied to the package were not recognized</p> </blockquote> <p>You will need to add yourself to the list of users who can access the private key (bring up the Dialog below via</p> <blockquote> <p>mmc - > Add/Remove Snap-in -> Certificates -> Local Computer</p> </blockquote> <p><img src="https://wallaceturner.azurewebsites.net/get/manage_private_keys.png" alt="manage<em>private</em>keys.png" /></p> <p>For the purposes of this article and to follow along with the video covering this article I will remove all permissions then re-add the users as required.</p> <p>You'll need to add yourself (output of <code>whoami</code>) to this list in order to get the code working when running via Visual Studio.</p> <h3>Configuration when running under IIS</h3> <p>First publish the app via</p> <pre><code>dotnet publish -c Release -r win10-x64 </code></pre> <p>We assume you have IIS installed and have modified the 'Default Web Site' physical path to point at the Release output folder (e.g. C:\temp\ravendemo\bin\Release\netcoreapp2.2\win10-x64\publish) Turn logging on in the <code>web.config</code> file by setting `stdoutLogEnabled="true"</p> <p>Bring up <code>http://localhost</code> (do not use port 5000 which is the dev environment; you want to hit IIS) You will likely get this unhelpful error message, even whilst running on localhost.</p> <blockquote> <p>An error occurred while starting the application. .NET Core 4.6.27617.05 X64 v4.0.0.0 | Microsoft.AspNetCore.Hosting version 2.2.0-rtm-35687 | Microsoft Windows 10.0.17763 | Need help?</p> </blockquote> <p>Open the logs file (C:\temp\ravendemo\bin\Release\netcoreapp2.2\win10-x64\publish\logs\stdout_zzz.log ) which reveals</p> <blockquote> <p>System.ComponentModel.Win32Exception: The credentials supplied to the package were not recognized</p> </blockquote> <p>Add IIS_IUSRS to the list of users using 'Manage Private Keys' as above:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/add_iis_user.png" alt="add<em>iis</em>user.png" /></p> <p><strong>Be sure to Recycle the App Pool</strong> (DefaultAppPool) and reload <code>http://localhost</code></p> <p>The page should successfully load and you will see a new Order in the RavenDb Studio.</p> <h3>Configuration when running a Windows Service</h3> <p>Modify your <code>Startup.cs</code> so you can run the webapp as a windows service <a href="https://gist.github.com/wallaceturner/2cf4bfe0ca805a109a344f8a282ab45b">see gist</a></p> <p>To install the service run</p> <pre><code>sc.exe create a_ravendemo binPath= "C:\temp\ravendemo\bin\Release\netcoreapp2.2\win10-x64\publish\ravendemo.exe --service" </code></pre> <p>Ensure the user whom the service is running under has access to the private key. If this is the default 'Local System Account' then you need to add the SYSTEM user instead of IIS_IUSRS as above.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/ravendb-4-part-2-client-configuration" /> 2019-05-23T04:10:25Z 2019-05-27T05:39:00Z Wallace Turner <p>Part 1 of this article can be found <a href="https://wallaceturner.azurewebsites.net/ravendb-4-tryout-part-1-installation">here</a></p> <p>View the <a href="https://www.youtube.com/watch?v=Yg_XAF6NJgM">youtube</a> walkthrough of this post which contains a few extra points of interest.</p> <p>Architecture Diagram</p> <p><img src="https://wallaceturner.azurewebsites.net/get/raven-network.png" alt="mraven-network.png" /></p> <p>Now we have our RavenDb servers set up lets create a web app from scratch that will communicate with the RavenDb cluster.</p> <pre><code>dotnet new webapp -o ravendemo dotnet add package RavenDB.Client --version 4.1.5 dotnet add package Microsoft.AspNetCore.Hosting.WindowsServices </code></pre> <p>Lets now add some code to configure the RavenDb <code>DocumentStore</code> singleton. Be sure to copy <code>client.pfx</code> to the output directory. For brevity the definition of the `Order class has been omitted.</p> <pre><code>var store = new DocumentStore { Urls = new[] { "https://raven1.mooo.com:8080" }, Database="TestDb"}; store.Conventions.IdentityPartsSeparator = "-"; X509Certificate2 certificate = new X509Certificate2(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "client.pfx")); store.Certificate = certificate; store.Initialize(); using(var session = store.OpenSession()) { session.Store(new Order { Name=Guid.NewGuid().ToString()}); session.SaveChanges(); } </code></pre> <p>When you run this you receive an error</p> <blockquote> <p>The SSL connection could not be established, see inner exception. -> The remote certificate is invalid according to the validation procedure.'</p> </blockquote> <p>The certificate is not trusted as it is signed by an untrusted CA. Fix this by importing client.pfx which also contains the CA certificate. Note that this time we will import the certificate into <code>Local Computer</code> by omitting the <code>-user</code> flag. You will thus need to run the following in an Admin Powershell prompt:</p> <pre><code>certutil -f -importpfx z:\client.pfx </code></pre> <p>Re-run the app in Visual Studio Code and it should work this time. This approach has a drawback in that <code>client.pfx</code> must be used and the password exposed potentially in a config file. An alternative is to use the Windows Certificate Store to grant access to the private key. Remove the code loading the client.pfx from a file and replace with:</p> <pre><code>var x509 = new X509Store(StoreLocation.LocalMachine); x509.Open(OpenFlags.ReadOnly); var certificate = x509.Certificates.Find(X509FindType.FindBySubjectName, "*.mooo.com", true)[0]; </code></pre> <p>If you re-run the app it will <em>most</em> likely work. This is due to the fact that by default the Administrators will have access to the private keys of the cert that was imported prior. If you are running in a more locked down environment (or clear out the users in the dialog shown below) you get this error:</p> <blockquote> <p>System.ComponentModel.Win32Exception: The credentials supplied to the package were not recognized</p> </blockquote> <p>You will need to add yourself to the list of users who can access the private key (bring up the Dialog below via</p> <blockquote> <p>mmc - > Add/Remove Snap-in -> Certificates -> Local Computer</p> </blockquote> <p><img src="https://wallaceturner.azurewebsites.net/get/manage_private_keys.png" alt="manage<em>private</em>keys.png" /></p> <p>For the purposes of this article and to follow along with the video covering this article I will remove all permissions then re-add the users as required.</p> <p>You'll need to add yourself (output of <code>whoami</code>) to this list in order to get the code working when running via Visual Studio.</p> <h3>Configuration when running under IIS</h3> <p>First publish the app via</p> <pre><code>dotnet publish -c Release -r win10-x64 </code></pre> <p>We assume you have IIS installed and have modified the 'Default Web Site' physical path to point at the Release output folder (e.g. C:\temp\ravendemo\bin\Release\netcoreapp2.2\win10-x64\publish) Turn logging on in the <code>web.config</code> file by setting `stdoutLogEnabled="true"</p> <p>Bring up <code>http://localhost</code> (do not use port 5000 which is the dev environment; you want to hit IIS) You will likely get this unhelpful error message, even whilst running on localhost.</p> <blockquote> <p>An error occurred while starting the application. .NET Core 4.6.27617.05 X64 v4.0.0.0 | Microsoft.AspNetCore.Hosting version 2.2.0-rtm-35687 | Microsoft Windows 10.0.17763 | Need help?</p> </blockquote> <p>Open the logs file (C:\temp\ravendemo\bin\Release\netcoreapp2.2\win10-x64\publish\logs\stdout_zzz.log ) which reveals</p> <blockquote> <p>System.ComponentModel.Win32Exception: The credentials supplied to the package were not recognized</p> </blockquote> <p>Add IIS_IUSRS to the list of users using 'Manage Private Keys' as above:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/add_iis_user.png" alt="add<em>iis</em>user.png" /></p> <p><strong>Be sure to Recycle the App Pool</strong> (DefaultAppPool) and reload <code>http://localhost</code></p> <p>The page should successfully load and you will see a new Order in the RavenDb Studio.</p> <h3>Configuration when running a Windows Service</h3> <p>Modify your <code>Startup.cs</code> so you can run the webapp as a windows service <a href="https://gist.github.com/wallaceturner/2cf4bfe0ca805a109a344f8a282ab45b">see gist</a></p> <p>To install the service run</p> <pre><code>sc.exe create a_ravendemo binPath= "C:\temp\ravendemo\bin\Release\netcoreapp2.2\win10-x64\publish\ravendemo.exe --service" </code></pre> <p>Ensure the user whom the service is running under has access to the private key. If this is the default 'Local System Account' then you need to add the SYSTEM user instead of IIS_IUSRS as above.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/ravendb-4-part-2-client-configuration" /> https://wallaceturner.azurewebsites.net/ravendb-4-part-1-installation RavenDb 4 Part 1 - Installation <p>Part 2 of this article can be found <a href="http://wallaceturner.com/ravendb-4-part-2-client-configuration">here</a></p> <p>View the <a href="https://www.youtube.com/watch?v=y4OF1dkOYI8">youtube</a> walkthrough of this post which contains a few extra points of interest.</p> <p>This article is intended for those setting up RavenDb in a secured (non-developer) environment using x509 certificates for mutual authentication. This is a manual configuration of RavenDb that uses self-signed certificates.</p> <p>A diagram of the architecture for the purposes of this discussion is shown below.</p> <p><em>Please note that RavenDb4 <a href="https://ravendb.net/docs/article-page/4.0/csharp/server/clustering/cluster-best-practice-and-configuration">recommends a minimum of 3 nodes</a> in a cluster however we'll use 2 and demonstrate some issues you might encounter.</em></p> <p><img src="https://wallaceturner.azurewebsites.net/get/raven-network.png" alt="mraven-network.png" /></p> <h3>Generate wildcard certificates</h3> <p>If you have an existing certificate (signed by CA or other) then you can skip this step.</p> <p>Start an Administrator Powershell prompt - this step can be performed on any server.</p> <p>Download the script used to generate the certificates - this script is from the RavenDb github repository.</p> <pre><code>curl.exe -LJO https://raw.githubusercontent.com/ravendb/ravendb/v4.1/scripts/certificates/powershell/generate-server-cert.ps1 </code></pre> <p>Create two certificates:</p> <pre><code>.\generate-server-cert.ps1 -CN *.mooo.com -CertFile server.pfx .\generate-server-cert.ps1 -CN *.mooo.com -CertFile client.pfx </code></pre> <h3>RavenDb Server Installation</h3> <p>Follow the steps below for each instance of RavenDb e.g. servers raven1, raven2 and raven3 in this example.</p> <p>Extract the <a href="https://ravendb.net/download">RavenDb zip</a> to a folder of your choice e.g. e.g. C:\RavenDB-4.2.0-windows-x64\</p> <p>Copy server.pfx &amp; client.pfx to RavenDb Server folder (e.g. C:\RavenDB-4.2.0-windows-x64\Server)</p> <p>In the RavenDb Server folder rename <code>settings.default.json</code> to <code>settings.json</code> and replace the contents with: </p> <p>replace <code>raven1.mooo.com</code> with the relevant domain - typically this will be the server name + domain name from the certificate</p> <pre><code>{ "DataDir": "RavenData", "License.Eula.Accepted": true, "Setup.Mode": "Secured", "Security.Certificate.Path": "server.pfx", "Security.Certificate.Password": "", "ServerUrl": "https://0.0.0.0:8080", "ServerUrl.Tcp": "tcp://0.0.0.0:38888", "PublicServerUrl": "https://raven1.mooo.com:8080", "PublicServerUrl.Tcp": "tcp://raven1.mooo.com:38888" } </code></pre> <p>Ensure DNS exists for your domains. (e.g. raven1.mooo.com) - Use the Windows hosts file if you want to try it out quickly.</p> <h3>Start RavenDb</h3> <p>Using a Powershell console navigate to base RavenDb folder (C:\RavenDB-4.2.0-windows-x64) and run <code>run.ps1</code></p> <h3>Import certificate into the RavenDb &amp; the Windows certificate store</h3> <p>Using the RavenDb console run</p> <pre><code>trustClientCert client C:\RavenDB-4.2.0-windows-x64\Server\client.pfx </code></pre> <p>The output should look something like</p> <blockquote> <p>Successfully read certificate: B4885D6E12C4D64136C333EDDFFF8540BF803688</p> </blockquote> <p>You can now delete client.pfx</p> <p>Using your Powershell console navigate to base RavenDb Server folder (C:\RavenDB-4.2.0-windows-x64\Server) and run</p> <pre><code>certutil -f -importpfx -user .\client.pfx </code></pre> <p>This step is necessary as it will allow the logged in user (you) to authenticate with the RavenDb instance.</p> <h3>Checking the installation</h3> <p>Navigate using Chrome to</p> <pre><code>https://raven1.mooo.com:8080 </code></pre> <p>If all goes well you should be prompted to select a certificate for authentication.</p> <p>Thats it! You have now set up your RavenDb instance securely. </p> <p>Repeat this process on raven2 and raven3 and then check out the <a href="https://www.youtube.com/watch?v=y4OF1dkOYI8">video</a> which demonstrates how to create a cluster and setup external replication.</p> <p>Check out <a href="https://wallaceturner.azurewebsites.net/ravendb-4-tryout-part-2-client-configuration">Part 2</a> which discusses how to get your dotnet webapp or service to talk to RavenDb.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/ravendb-4-part-1-installation" /> 2019-05-22T03:19:00Z 2019-05-27T05:22:54Z Wallace Turner <p>Part 2 of this article can be found <a href="http://wallaceturner.com/ravendb-4-part-2-client-configuration">here</a></p> <p>View the <a href="https://www.youtube.com/watch?v=y4OF1dkOYI8">youtube</a> walkthrough of this post which contains a few extra points of interest.</p> <p>This article is intended for those setting up RavenDb in a secured (non-developer) environment using x509 certificates for mutual authentication. This is a manual configuration of RavenDb that uses self-signed certificates.</p> <p>A diagram of the architecture for the purposes of this discussion is shown below.</p> <p><em>Please note that RavenDb4 <a href="https://ravendb.net/docs/article-page/4.0/csharp/server/clustering/cluster-best-practice-and-configuration">recommends a minimum of 3 nodes</a> in a cluster however we'll use 2 and demonstrate some issues you might encounter.</em></p> <p><img src="https://wallaceturner.azurewebsites.net/get/raven-network.png" alt="mraven-network.png" /></p> <h3>Generate wildcard certificates</h3> <p>If you have an existing certificate (signed by CA or other) then you can skip this step.</p> <p>Start an Administrator Powershell prompt - this step can be performed on any server.</p> <p>Download the script used to generate the certificates - this script is from the RavenDb github repository.</p> <pre><code>curl.exe -LJO https://raw.githubusercontent.com/ravendb/ravendb/v4.1/scripts/certificates/powershell/generate-server-cert.ps1 </code></pre> <p>Create two certificates:</p> <pre><code>.\generate-server-cert.ps1 -CN *.mooo.com -CertFile server.pfx .\generate-server-cert.ps1 -CN *.mooo.com -CertFile client.pfx </code></pre> <h3>RavenDb Server Installation</h3> <p>Follow the steps below for each instance of RavenDb e.g. servers raven1, raven2 and raven3 in this example.</p> <p>Extract the <a href="https://ravendb.net/download">RavenDb zip</a> to a folder of your choice e.g. e.g. C:\RavenDB-4.2.0-windows-x64\</p> <p>Copy server.pfx &amp; client.pfx to RavenDb Server folder (e.g. C:\RavenDB-4.2.0-windows-x64\Server)</p> <p>In the RavenDb Server folder rename <code>settings.default.json</code> to <code>settings.json</code> and replace the contents with: </p> <p>replace <code>raven1.mooo.com</code> with the relevant domain - typically this will be the server name + domain name from the certificate</p> <pre><code>{ "DataDir": "RavenData", "License.Eula.Accepted": true, "Setup.Mode": "Secured", "Security.Certificate.Path": "server.pfx", "Security.Certificate.Password": "", "ServerUrl": "https://0.0.0.0:8080", "ServerUrl.Tcp": "tcp://0.0.0.0:38888", "PublicServerUrl": "https://raven1.mooo.com:8080", "PublicServerUrl.Tcp": "tcp://raven1.mooo.com:38888" } </code></pre> <p>Ensure DNS exists for your domains. (e.g. raven1.mooo.com) - Use the Windows hosts file if you want to try it out quickly.</p> <h3>Start RavenDb</h3> <p>Using a Powershell console navigate to base RavenDb folder (C:\RavenDB-4.2.0-windows-x64) and run <code>run.ps1</code></p> <h3>Import certificate into the RavenDb &amp; the Windows certificate store</h3> <p>Using the RavenDb console run</p> <pre><code>trustClientCert client C:\RavenDB-4.2.0-windows-x64\Server\client.pfx </code></pre> <p>The output should look something like</p> <blockquote> <p>Successfully read certificate: B4885D6E12C4D64136C333EDDFFF8540BF803688</p> </blockquote> <p>You can now delete client.pfx</p> <p>Using your Powershell console navigate to base RavenDb Server folder (C:\RavenDB-4.2.0-windows-x64\Server) and run</p> <pre><code>certutil -f -importpfx -user .\client.pfx </code></pre> <p>This step is necessary as it will allow the logged in user (you) to authenticate with the RavenDb instance.</p> <h3>Checking the installation</h3> <p>Navigate using Chrome to</p> <pre><code>https://raven1.mooo.com:8080 </code></pre> <p>If all goes well you should be prompted to select a certificate for authentication.</p> <p>Thats it! You have now set up your RavenDb instance securely. </p> <p>Repeat this process on raven2 and raven3 and then check out the <a href="https://www.youtube.com/watch?v=y4OF1dkOYI8">video</a> which demonstrates how to create a cluster and setup external replication.</p> <p>Check out <a href="https://wallaceturner.azurewebsites.net/ravendb-4-tryout-part-2-client-configuration">Part 2</a> which discusses how to get your dotnet webapp or service to talk to RavenDb.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/ravendb-4-part-1-installation" /> https://wallaceturner.azurewebsites.net/run-asp-net-core-web-app-as-a-windows-service-via-iis Run asp.net core web app as a windows service via IIS <p>Rick Strahl has a <a href="https://weblog.west-wind.com/posts/2016/Jun/06/Publishing-and-Running-ASPNET-Core-Applications-with-IIS">thorough article</a> on publishing your dot net core app with IIS</p> <p>This requires you to install the <code>AspNetCoreModule</code> into IIS and it is this module which manages the lifetime of your dot net core process which runs separately to <code>w3wp.exe</code> in its own process (e.g. <code>WebApplication1.exe</code>)</p> <p>This is less than ideal if your web app runs background tasks that you want running constantly - it requires you to poll your website after a server or IIS reboot to ensure your process is loaded and running (via <code>AspNetCoreModule</code>)</p> <p>It is possible to host the dotnet core app as a windows service but this wont work with AspNetCoreModule - the latter will simply ignore the running process and start its own new instance.</p> <p>To get this working so you have your dotnet core app running as a service <em>and</em> have it hosted via IIS (importantly on ports 80,443 with other hosted web apps) the following steps are required: (assumes you are starting from a <code>File -&gt; New Project -&gt; ASP.NET Core Web Application</code>)</p> <p>1) in <code>project.json</code> add references to</p> <pre><code>"Microsoft.AspNetCore.Hosting": "1.0.0-*", "Microsoft.AspNetCore.Hosting.WindowsServices": "1.0.0-*", </code></pre> <p>This will result in an error </p> <blockquote> <p>The dependency Microsoft.AspNetCore.Hosting.WindowsServices 1.0.0 does not support framework .NETCoreApp,Version=v1.0.</p> </blockquote> <p>You need to target the full framework - this suited my purposes fine as i was running on windows only and depended on libraries not yet supported by dotnet core. If you absoutely must have this working on other platforms as a pure dotnet core app then check out <a href="https://github.com/dasMulli/dotnet-win32-service">dasMulli/dotnet-win32-service</a></p> <p>Replace the entire <code>frameworks</code> section in <code>project.json</code> with</p> <pre><code>"frameworks": { "net461": { } }, </code></pre> <p>and <em>remove</em> from <code>dependencies</code> this section</p> <pre><code>"Microsoft.NETCore.App": { "version": "1.0.1", "type": "platform" } </code></pre> <p>2) Modify your <code>Program.cs</code> file:</p> <pre><code>if (Debugger.IsAttached || args.Contains("--debug")) { host.Run(); } else { host.RunAsService();//extension method in Microsoft.AspNetCore.Hosting.WindowsServices } </code></pre> <p>3) Publish your web app to the file system and navigate to the output folder (e.g. <code>WebApplication1\src\WebApplication1\bin\Release\PublishOutput</code>)</p> <p>4) Install and run app as a service (from admin command prompt)</p> <pre><code>sc create WebApplication1 binpath="full path to your exe" </code></pre> <p>You should now be able to navigate and test your web app via <code>http://localhost:5000/api/values</code></p> <p>5) Lastly you need to setup a proxy in IIS to route requests to your windows service: This step is largely copied from <a href="https://weblogs.asp.net/owscott/creating-a-reverse-proxy-with-url-rewrite-for-iis">this article</a> by Scott Forsyth</p> <p>Create an application pool for the website that is <code>No Managed Code</code> (runtime not required for a simply proxy)</p> <p><strong>Do not point at your dotnet core output folder</strong> Instead just point it at an empty folder that IIS has access to. This folder will simply contain a <code>web.config</code> file with the rewrite rule. (don't use a temp folder in production either :))</p> <p><img src="http://wallaceturner.com/get/MWSnap0010%202017-02-05%2c%2006_59_10.png" alt="alt text" /></p> <p>Install <a href="https://www.iis.net/downloads/microsoft/url-rewrite">Url Rewrite</a> into IIS and add a reverse proxy</p> <p><img src="http://wallaceturner.com/get/reverse_proxy1.png" alt="alt text" /></p> <p>n.b. do <strong>not</strong> add <code>http</code> to the start of the address <img src="http://wallaceturner.com/get/reverse_proxy2.png" alt="alt text" /></p> <p>Start your website and navigate to <code>http://localhost/api/values</code> - hopefully it all works!</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/run-asp-net-core-web-app-as-a-windows-service-via-iis" /> 2017-02-05T07:07:00Z 2017-03-08T07:07:56Z Wallace Turner <p>Rick Strahl has a <a href="https://weblog.west-wind.com/posts/2016/Jun/06/Publishing-and-Running-ASPNET-Core-Applications-with-IIS">thorough article</a> on publishing your dot net core app with IIS</p> <p>This requires you to install the <code>AspNetCoreModule</code> into IIS and it is this module which manages the lifetime of your dot net core process which runs separately to <code>w3wp.exe</code> in its own process (e.g. <code>WebApplication1.exe</code>)</p> <p>This is less than ideal if your web app runs background tasks that you want running constantly - it requires you to poll your website after a server or IIS reboot to ensure your process is loaded and running (via <code>AspNetCoreModule</code>)</p> <p>It is possible to host the dotnet core app as a windows service but this wont work with AspNetCoreModule - the latter will simply ignore the running process and start its own new instance.</p> <p>To get this working so you have your dotnet core app running as a service <em>and</em> have it hosted via IIS (importantly on ports 80,443 with other hosted web apps) the following steps are required: (assumes you are starting from a <code>File -&gt; New Project -&gt; ASP.NET Core Web Application</code>)</p> <p>1) in <code>project.json</code> add references to</p> <pre><code>"Microsoft.AspNetCore.Hosting": "1.0.0-*", "Microsoft.AspNetCore.Hosting.WindowsServices": "1.0.0-*", </code></pre> <p>This will result in an error </p> <blockquote> <p>The dependency Microsoft.AspNetCore.Hosting.WindowsServices 1.0.0 does not support framework .NETCoreApp,Version=v1.0.</p> </blockquote> <p>You need to target the full framework - this suited my purposes fine as i was running on windows only and depended on libraries not yet supported by dotnet core. If you absoutely must have this working on other platforms as a pure dotnet core app then check out <a href="https://github.com/dasMulli/dotnet-win32-service">dasMulli/dotnet-win32-service</a></p> <p>Replace the entire <code>frameworks</code> section in <code>project.json</code> with</p> <pre><code>"frameworks": { "net461": { } }, </code></pre> <p>and <em>remove</em> from <code>dependencies</code> this section</p> <pre><code>"Microsoft.NETCore.App": { "version": "1.0.1", "type": "platform" } </code></pre> <p>2) Modify your <code>Program.cs</code> file:</p> <pre><code>if (Debugger.IsAttached || args.Contains("--debug")) { host.Run(); } else { host.RunAsService();//extension method in Microsoft.AspNetCore.Hosting.WindowsServices } </code></pre> <p>3) Publish your web app to the file system and navigate to the output folder (e.g. <code>WebApplication1\src\WebApplication1\bin\Release\PublishOutput</code>)</p> <p>4) Install and run app as a service (from admin command prompt)</p> <pre><code>sc create WebApplication1 binpath="full path to your exe" </code></pre> <p>You should now be able to navigate and test your web app via <code>http://localhost:5000/api/values</code></p> <p>5) Lastly you need to setup a proxy in IIS to route requests to your windows service: This step is largely copied from <a href="https://weblogs.asp.net/owscott/creating-a-reverse-proxy-with-url-rewrite-for-iis">this article</a> by Scott Forsyth</p> <p>Create an application pool for the website that is <code>No Managed Code</code> (runtime not required for a simply proxy)</p> <p><strong>Do not point at your dotnet core output folder</strong> Instead just point it at an empty folder that IIS has access to. This folder will simply contain a <code>web.config</code> file with the rewrite rule. (don't use a temp folder in production either :))</p> <p><img src="http://wallaceturner.com/get/MWSnap0010%202017-02-05%2c%2006_59_10.png" alt="alt text" /></p> <p>Install <a href="https://www.iis.net/downloads/microsoft/url-rewrite">Url Rewrite</a> into IIS and add a reverse proxy</p> <p><img src="http://wallaceturner.com/get/reverse_proxy1.png" alt="alt text" /></p> <p>n.b. do <strong>not</strong> add <code>http</code> to the start of the address <img src="http://wallaceturner.com/get/reverse_proxy2.png" alt="alt text" /></p> <p>Start your website and navigate to <code>http://localhost/api/values</code> - hopefully it all works!</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/run-asp-net-core-web-app-as-a-windows-service-via-iis" /> https://wallaceturner.azurewebsites.net/add-sass-and-autoprefixer-to-gulp-in-visual-studio-2015 Add sass and autoprefixer to gulp in visual studio 2015 <p>In Visual studio 2015 (Update 2) a new ASP.NET Core Web Application contains a gulpfile that adds css and js minification. Here is how to add scss and autoprefixing support to that gulpfile.</p> <p>After adding a new project rename <code>site.css</code> to <code>site.scss</code></p> <p><img src="http://wallaceturner.com/get/css_rename.jpg" alt="alt text" /></p> <p>Add something to the scss which will be later parsed by autoprefixer and sass, eg</p> <pre><code>$blue: #3bbfce; .content-navigation { display: flex; border-color: $blue; } </code></pre> <p>Next install the the required npm packages:</p> <pre><code>npm install gulp-autoprefixer npm install gulp-sass </code></pre> <p>You need to ensure you run these commands from the project folder (not just anywhere) eg for me that means opening a command prompt at</p> <pre><code> D:\aaa_development\spikes\GulpSpike\src\GulpSpike </code></pre> <p>Next, attempt to reference these from <code>gulpfile.js</code> by adding the following lines after the other existing require definitions</p> <pre><code>var autoprefixer = require('gulp-autoprefixer') var sass = require('gulp-sass'); </code></pre> <p>Run the gulpfile via the Task Runner Explorer (View->Other Windows)</p> <p><img src="http://wallaceturner.com/get/task_runner_explorer.jpg" alt="alt text" /></p> <p>An error occurs:</p> <blockquote> <p>Failed to run "D:\aaa<em>development\spikes\GulpSpike\src\GulpSpike\Gulpfile.js"... cmd.exe /c gulp --tasks-simple D:\aaa</em>development\spikes\GulpSpike\src\GulpSpike\node<em>modules\gulp-sass\node</em>modules\node-sass\lib\index.js:14 throw new Error(errors.missingBinary()); ^ Error: Missing binding D:\aaa<em>development\spikes\GulpSpike\src\GulpSpike\node</em>modules\gulp-sass\node_modules\node-sass\vendor\win32-ia32-47\binding.node Node Sass could not find a binding for your current environment: Windows 32-bit with Node.js 5.x Found bindings for the following environments: - Windows 64-bit with Node 0.10.x This usually happens because your environment has changed since running <code>npm install</code>. Run <code>npm rebuild node-sass</code> to build the binding for your current environment.</p> </blockquote> <p>Their suggested fix did not work. For some reason on the install the appropriate binding.node was not downloaded so I had to do this part manually. Go to the <a href="https://github.com/sass/node-sass/releases">node-sass releases page</a> and download the file the error message refers to, in this case <code>win32-ia32-47.binding.node</code> - rename the file to <code>binding.node</code> and place it in a new folder according to the error message</p> <blockquote> <p>D:\aaa<em>development\spikes\GulpSpike\src\GulpSpike\node</em>modules\gulp-sass\node_modules\node-sass\vendor\win32-ia32-47</p> </blockquote> <p>Now you can refresh the gulpfile and that error should no longer occur. Now to add the autoprefixer and sass gulp tasks:</p> <pre><code>gulp.task('sass', function () { return gulp.src('./wwwroot/css/**/*.scss') .pipe(sass().on('error', sass.logError)) .pipe(autoprefixer()) .pipe(gulp.dest('./wwwroot/css')); }); </code></pre> <p>Run the <code>sass</code> gulp task by right clicking on it in the Task Runner and selecting <code>Run</code>.</p> <p>Navigate to the output folder (which in this case is wwwroot\css, the same folder as the scss file) and you should see site.css has been created and instead of the scss you will hopefully see</p> <pre><code>.content-navigation { display: -webkit-box; display: -ms-flexbox; display: flex; border-color: #3bbfce; } </code></pre> <p>That is, autoprefixer has done its thing on <code>display: flex</code> and sass has replaced <code>border-color</code>.</p> <p>You will likely want the sass gulp task to run <em>before</em> css minification task. To do that modify the <code>min:css</code> task from</p> <pre><code>gulp.task("min:css", function () { </code></pre> <p>to</p> <pre><code>gulp.task("min:css", ['sass'], function () { </code></pre> <p>Lastly set up a watch so you can change the scss file in VS and see the changes in the browser without having to manually run the gulp task</p> <pre><code>gulp.task('watch', function () { return gulp .watch('./wwwroot/css/**/*.scss', ['sass']) .on('change', function (event) { console.log('File ' + event.path + ' was ' + event.type + ', running tasks...'); }); }); </code></pre> <p>One nice thing is the browser can automatically update when you change the scss file however I have found you must press <code>Ctrl-Alt-Enter</code> instead of simply saving the file to refresh any connected browsers.</p> <p>Download the final <a href="http://wallaceturner.com/get/gulpfile.js">gulpfile.js here</a></p> <p>References: </p> <p><a href="https://www.sitepoint.com/simple-gulpy-workflow-sass/">https://www.sitepoint.com/simple-gulpy-workflow-sass/</a></p> <img src="https://wallaceturner.azurewebsites.net/via-feed/add-sass-and-autoprefixer-to-gulp-in-visual-studio-2015" /> 2016-05-21T03:08:01Z 2016-05-21T05:17:02Z Wallace Turner <p>In Visual studio 2015 (Update 2) a new ASP.NET Core Web Application contains a gulpfile that adds css and js minification. Here is how to add scss and autoprefixing support to that gulpfile.</p> <p>After adding a new project rename <code>site.css</code> to <code>site.scss</code></p> <p><img src="http://wallaceturner.com/get/css_rename.jpg" alt="alt text" /></p> <p>Add something to the scss which will be later parsed by autoprefixer and sass, eg</p> <pre><code>$blue: #3bbfce; .content-navigation { display: flex; border-color: $blue; } </code></pre> <p>Next install the the required npm packages:</p> <pre><code>npm install gulp-autoprefixer npm install gulp-sass </code></pre> <p>You need to ensure you run these commands from the project folder (not just anywhere) eg for me that means opening a command prompt at</p> <pre><code> D:\aaa_development\spikes\GulpSpike\src\GulpSpike </code></pre> <p>Next, attempt to reference these from <code>gulpfile.js</code> by adding the following lines after the other existing require definitions</p> <pre><code>var autoprefixer = require('gulp-autoprefixer') var sass = require('gulp-sass'); </code></pre> <p>Run the gulpfile via the Task Runner Explorer (View->Other Windows)</p> <p><img src="http://wallaceturner.com/get/task_runner_explorer.jpg" alt="alt text" /></p> <p>An error occurs:</p> <blockquote> <p>Failed to run "D:\aaa<em>development\spikes\GulpSpike\src\GulpSpike\Gulpfile.js"... cmd.exe /c gulp --tasks-simple D:\aaa</em>development\spikes\GulpSpike\src\GulpSpike\node<em>modules\gulp-sass\node</em>modules\node-sass\lib\index.js:14 throw new Error(errors.missingBinary()); ^ Error: Missing binding D:\aaa<em>development\spikes\GulpSpike\src\GulpSpike\node</em>modules\gulp-sass\node_modules\node-sass\vendor\win32-ia32-47\binding.node Node Sass could not find a binding for your current environment: Windows 32-bit with Node.js 5.x Found bindings for the following environments: - Windows 64-bit with Node 0.10.x This usually happens because your environment has changed since running <code>npm install</code>. Run <code>npm rebuild node-sass</code> to build the binding for your current environment.</p> </blockquote> <p>Their suggested fix did not work. For some reason on the install the appropriate binding.node was not downloaded so I had to do this part manually. Go to the <a href="https://github.com/sass/node-sass/releases">node-sass releases page</a> and download the file the error message refers to, in this case <code>win32-ia32-47.binding.node</code> - rename the file to <code>binding.node</code> and place it in a new folder according to the error message</p> <blockquote> <p>D:\aaa<em>development\spikes\GulpSpike\src\GulpSpike\node</em>modules\gulp-sass\node_modules\node-sass\vendor\win32-ia32-47</p> </blockquote> <p>Now you can refresh the gulpfile and that error should no longer occur. Now to add the autoprefixer and sass gulp tasks:</p> <pre><code>gulp.task('sass', function () { return gulp.src('./wwwroot/css/**/*.scss') .pipe(sass().on('error', sass.logError)) .pipe(autoprefixer()) .pipe(gulp.dest('./wwwroot/css')); }); </code></pre> <p>Run the <code>sass</code> gulp task by right clicking on it in the Task Runner and selecting <code>Run</code>.</p> <p>Navigate to the output folder (which in this case is wwwroot\css, the same folder as the scss file) and you should see site.css has been created and instead of the scss you will hopefully see</p> <pre><code>.content-navigation { display: -webkit-box; display: -ms-flexbox; display: flex; border-color: #3bbfce; } </code></pre> <p>That is, autoprefixer has done its thing on <code>display: flex</code> and sass has replaced <code>border-color</code>.</p> <p>You will likely want the sass gulp task to run <em>before</em> css minification task. To do that modify the <code>min:css</code> task from</p> <pre><code>gulp.task("min:css", function () { </code></pre> <p>to</p> <pre><code>gulp.task("min:css", ['sass'], function () { </code></pre> <p>Lastly set up a watch so you can change the scss file in VS and see the changes in the browser without having to manually run the gulp task</p> <pre><code>gulp.task('watch', function () { return gulp .watch('./wwwroot/css/**/*.scss', ['sass']) .on('change', function (event) { console.log('File ' + event.path + ' was ' + event.type + ', running tasks...'); }); }); </code></pre> <p>One nice thing is the browser can automatically update when you change the scss file however I have found you must press <code>Ctrl-Alt-Enter</code> instead of simply saving the file to refresh any connected browsers.</p> <p>Download the final <a href="http://wallaceturner.com/get/gulpfile.js">gulpfile.js here</a></p> <p>References: </p> <p><a href="https://www.sitepoint.com/simple-gulpy-workflow-sass/">https://www.sitepoint.com/simple-gulpy-workflow-sass/</a></p> <img src="https://wallaceturner.azurewebsites.net/via-feed/add-sass-and-autoprefixer-to-gulp-in-visual-studio-2015" /> https://wallaceturner.azurewebsites.net/in-praise-of-ravendb in praise of ravendb <p>Recently I had to re-install my OS. Part of that job required reinstalling SQL server express 2014 and ravendb. How long did each take?</p> <p>Raven took about 20 seconds</p> <p>Sql server took a lot longer - like a lot.</p> <p><strong>Even tho I am a developer by trade there are a few devops-style jobs that are required on a daily basis. The speed of each of these starts to form the basis of your opinion of the product itself.</strong></p> <h3>RavenDb</h3> <p>RavenDb (and all the existing databases) live on a partitioned drive so were not wiped out when the OS reinstalled. To re-install I just had to navigate to the Server folder and type</p> <pre><code>Raven.Server /install </code></pre> <p>to install ravendb as a service. now I can browse <code>http://localhost:8080</code> and all my databases are there, yay!</p> <h3>Sql Server</h3> <p>I had the installation files for both sql express and the management studio downloaded already so I went straight to the installer. Their installer is extremely clunky like it is made to irritate the installer. You cant simply set and forget but have to come back every 10-20 seconds and click next next it is like water boarding. By that i mean you click <code>Next</code> and you have to wait till the next panel of information loads, then likely hit <code>Next</code> again without changing anything.</p> <p>After doing this twice (once for core install and once for the mgmt studio) i realised i had installed as a named instance and all our connection strings use <code>(local)</code> not <code>\sqlexpress</code> aaaaand there is NO WAY to change to a default instance without re-installing! wtf?</p> <p>So i go to uninstall what I just did and then re-install and guess what? it tells me a reboot is required. more time wasting.</p> <p>After its installed you then need to re-import your databases back into the fresh install. (which is still in progress)</p> <p>Also just to add insult now in Add/Remove Programs instead of seeing an extra entry for 'sql server express 2014' i see a whole bunch of other kludge i never formally installed and would never likely need on its own:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/Ypxcf2q.png" alt="alt text" /></p> <p>Oh and after all that all you get is a relational database, BOO!</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/in-praise-of-ravendb" /> 2016-04-19T05:28:47Z 2016-04-19T05:38:26Z Wallace Turner <p>Recently I had to re-install my OS. Part of that job required reinstalling SQL server express 2014 and ravendb. How long did each take?</p> <p>Raven took about 20 seconds</p> <p>Sql server took a lot longer - like a lot.</p> <p><strong>Even tho I am a developer by trade there are a few devops-style jobs that are required on a daily basis. The speed of each of these starts to form the basis of your opinion of the product itself.</strong></p> <h3>RavenDb</h3> <p>RavenDb (and all the existing databases) live on a partitioned drive so were not wiped out when the OS reinstalled. To re-install I just had to navigate to the Server folder and type</p> <pre><code>Raven.Server /install </code></pre> <p>to install ravendb as a service. now I can browse <code>http://localhost:8080</code> and all my databases are there, yay!</p> <h3>Sql Server</h3> <p>I had the installation files for both sql express and the management studio downloaded already so I went straight to the installer. Their installer is extremely clunky like it is made to irritate the installer. You cant simply set and forget but have to come back every 10-20 seconds and click next next it is like water boarding. By that i mean you click <code>Next</code> and you have to wait till the next panel of information loads, then likely hit <code>Next</code> again without changing anything.</p> <p>After doing this twice (once for core install and once for the mgmt studio) i realised i had installed as a named instance and all our connection strings use <code>(local)</code> not <code>\sqlexpress</code> aaaaand there is NO WAY to change to a default instance without re-installing! wtf?</p> <p>So i go to uninstall what I just did and then re-install and guess what? it tells me a reboot is required. more time wasting.</p> <p>After its installed you then need to re-import your databases back into the fresh install. (which is still in progress)</p> <p>Also just to add insult now in Add/Remove Programs instead of seeing an extra entry for 'sql server express 2014' i see a whole bunch of other kludge i never formally installed and would never likely need on its own:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/Ypxcf2q.png" alt="alt text" /></p> <p>Oh and after all that all you get is a relational database, BOO!</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/in-praise-of-ravendb" /> https://wallaceturner.azurewebsites.net/nuget-cheat-sheet nuget cheat sheet <p>I'm constantly forgetting these simple commands as its not everyday you use the package manager console. Also, they arent the most obvious commands, especially the first two.</p> <p>List packages you have installed</p> <pre><code>Get-Package </code></pre> <p>List available packages containing the word foo</p> <pre><code>Get-Package -ListAvailable foo </code></pre> <p>Install package:</p> <pre><code>Install-Package foo </code></pre> <p>Uninstall or remove a package</p> <pre><code>Uninstall-Package foo -RemoveDependencies Uninstall-Package Newtonsoft.Json -Version 6.0.4 </code></pre> <p>Help</p> <pre><code>Get-Help Uninstall-Package </code></pre> <p>Add other commonly used ones in the comments section and I will update the list.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/nuget-cheat-sheet" /> 2013-02-21T01:22:00Z 2016-03-29T01:22:59Z Wallace Turner <p>I'm constantly forgetting these simple commands as its not everyday you use the package manager console. Also, they arent the most obvious commands, especially the first two.</p> <p>List packages you have installed</p> <pre><code>Get-Package </code></pre> <p>List available packages containing the word foo</p> <pre><code>Get-Package -ListAvailable foo </code></pre> <p>Install package:</p> <pre><code>Install-Package foo </code></pre> <p>Uninstall or remove a package</p> <pre><code>Uninstall-Package foo -RemoveDependencies Uninstall-Package Newtonsoft.Json -Version 6.0.4 </code></pre> <p>Help</p> <pre><code>Get-Help Uninstall-Package </code></pre> <p>Add other commonly used ones in the comments section and I will update the list.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/nuget-cheat-sheet" /> https://wallaceturner.azurewebsites.net/resharper-structural-search-and-replace-part-ii-adding-your-own-quick-fixes Resharper structural search and replace Part II - Custom Quick Fixes <p>In <a href="http://wallaceturner.com/resharper-structural-search-and-replace">Part I</a> I covered finding usages with Resharper's 'Search with Pattern' feature.</p> <p>Now lets turn our focus to replacing usages with our own snippets and creating our own custom quick fixes!</p> <p>Consider this erroneous code sample you might encounter:</p> <pre><code>private void HandleMessage(object data) { Address address = data as Address;//should just cast to Address here if(address.StreetName == "easy street") { //handle... } } </code></pre> <p>Why erroneous you say? Well if you expect <code>data</code> to <em>only</em> ever be an <code>Address</code> then you should cast to it instead of using the <code>as</code> operator - That way at least you'll get an <code>InvalidCastException</code> instead of a <code>NullPointerException</code>.<br /> <strong>sidenote1</strong> <em>there is obviously a time and place for the <code>as</code> operator I'm just giving an example of where a cast is more appropriate</em></p> <p>We'll add a pattern that will replace: </p> <pre><code>Address address = data as Address; </code></pre> <p>with </p> <pre><code>var address = (Address)data; </code></pre> <p>This time open the Patterns Catalog, <code>Resharper -&gt; Options -&gt; Code Inspection -&gt; Custom Patterns</code></p> <p><img src="https://wallaceturner.azurewebsites.net\get\custompatterns1.png" alt="alt text" /></p> <p>Click <code>Add Pattern</code> and enter the following details:</p> <p><img src="https://wallaceturner.azurewebsites.net\get\custompatterns2.png" alt="alt text" /></p> <p>Search for:</p> <pre><code>$type$ $var$ = $expr$ as $type$; </code></pre> <p>Replacing with:</p> <pre><code>var $var$ = ($type$)$expr$; </code></pre> <p>Now you can Save the Pattern and press 'Search Now' which will find our usage as detailed above and give you the option to Replace it (with undo naturally!)<br /> With this specific example you'd never want to Find and Replace All as there would be many valid usages of <code>as</code>; instead we would rather this pattern applied on a per-case basis: enter the real fun...</p> <h3>Your pattern as a Quick Fix</h3> <p>Put the cursor at the <strong>start</strong> of the line as shown and press the famous <code>Alt-Enter</code>:</p> <p><img src="https://wallaceturner.azurewebsites.net\get\custompatterns3.png" alt="alt text" /></p> <p>You now have your own quick fix! Since we set the pattern to 'Show as hint' it will only show when you place the cursor in the magical spot and won't underline or draw attention away from you. to reiterate: <strong>this quick fix is only applicable in the right situation</strong></p> <p><strong>sidenote2</strong> <em>seems like a lot of trouble to go to for something that can be fixed fairly easily manually?? depends who you've let loose on your code and how many times you have to replace it</em></p> <img src="https://wallaceturner.azurewebsites.net/via-feed/resharper-structural-search-and-replace-part-ii-adding-your-own-quick-fixes" /> 2012-02-21T12:04:11Z 2012-02-21T13:38:17Z Wallace Turner <p>In <a href="http://wallaceturner.com/resharper-structural-search-and-replace">Part I</a> I covered finding usages with Resharper's 'Search with Pattern' feature.</p> <p>Now lets turn our focus to replacing usages with our own snippets and creating our own custom quick fixes!</p> <p>Consider this erroneous code sample you might encounter:</p> <pre><code>private void HandleMessage(object data) { Address address = data as Address;//should just cast to Address here if(address.StreetName == "easy street") { //handle... } } </code></pre> <p>Why erroneous you say? Well if you expect <code>data</code> to <em>only</em> ever be an <code>Address</code> then you should cast to it instead of using the <code>as</code> operator - That way at least you'll get an <code>InvalidCastException</code> instead of a <code>NullPointerException</code>.<br /> <strong>sidenote1</strong> <em>there is obviously a time and place for the <code>as</code> operator I'm just giving an example of where a cast is more appropriate</em></p> <p>We'll add a pattern that will replace: </p> <pre><code>Address address = data as Address; </code></pre> <p>with </p> <pre><code>var address = (Address)data; </code></pre> <p>This time open the Patterns Catalog, <code>Resharper -&gt; Options -&gt; Code Inspection -&gt; Custom Patterns</code></p> <p><img src="https://wallaceturner.azurewebsites.net\get\custompatterns1.png" alt="alt text" /></p> <p>Click <code>Add Pattern</code> and enter the following details:</p> <p><img src="https://wallaceturner.azurewebsites.net\get\custompatterns2.png" alt="alt text" /></p> <p>Search for:</p> <pre><code>$type$ $var$ = $expr$ as $type$; </code></pre> <p>Replacing with:</p> <pre><code>var $var$ = ($type$)$expr$; </code></pre> <p>Now you can Save the Pattern and press 'Search Now' which will find our usage as detailed above and give you the option to Replace it (with undo naturally!)<br /> With this specific example you'd never want to Find and Replace All as there would be many valid usages of <code>as</code>; instead we would rather this pattern applied on a per-case basis: enter the real fun...</p> <h3>Your pattern as a Quick Fix</h3> <p>Put the cursor at the <strong>start</strong> of the line as shown and press the famous <code>Alt-Enter</code>:</p> <p><img src="https://wallaceturner.azurewebsites.net\get\custompatterns3.png" alt="alt text" /></p> <p>You now have your own quick fix! Since we set the pattern to 'Show as hint' it will only show when you place the cursor in the magical spot and won't underline or draw attention away from you. to reiterate: <strong>this quick fix is only applicable in the right situation</strong></p> <p><strong>sidenote2</strong> <em>seems like a lot of trouble to go to for something that can be fixed fairly easily manually?? depends who you've let loose on your code and how many times you have to replace it</em></p> <img src="https://wallaceturner.azurewebsites.net/via-feed/resharper-structural-search-and-replace-part-ii-adding-your-own-quick-fixes" /> https://wallaceturner.azurewebsites.net/resharper-structural-search-and-replace Resharper structural search and replace Part I - Searching <p>There isn't much documentation on Resharper's <a href="http://blogs.jetbrains.com/dotnet/2010/04/introducing-resharper-50-structural-">Structural Search and Replace</a> (SSR) so here is 2 such ways you might find it useful.</p> <p>Consider the case where you want to find all the usages of the <code>Text</code> property on a Windows <code>Form</code>:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/form_usages.png" alt="form_usages.png" /></p> <p>If you use Find Usages (Shift F12) you get <em>all</em> usages of any inheritor of <code>System.Windows.Forms.Control</code> which isn't what we want in this case.</p> <p><img src="https://wallaceturner.azurewebsites.net/get/usages1.png" alt="usages1.png" /></p> <p>Enter Resharpers 'Search with pattern' (Alt-R,F,R)</p> <p><img src="https://wallaceturner.azurewebsites.net/get/structural_search_and_replace2.png" alt="structural<em>search</em>and_replace2.png" /></p> <p>which returns the following usages only:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/usages2.png" alt="usages2.png" /></p> <p>I can hear the guys up the back:</p> <blockquote> <p>you can already do that with Resharper's Advanced Find Usages (Ctrl+Shift+Alt+F12) </p> </blockquote> <p>but this only works in this instance because <code>Text</code> is overridden by <code>System.Windows.Form</code>. Consider a property of <code>Form</code> that isn't overridden (eg <code>form.Font</code>) - your only option here is to use SSR.</p> <h3>Other applications</h3> <p>You are searching for places where an <code>Address</code> is added to a <code>List&lt;Address&gt;</code> </p> <pre><code>namespace ExampleApp { public class Address { } public class Program { private static void SomeMethod() { var strings = new List&lt;string&gt;(); strings.Add("test1"); strings.Add("test2"); strings.Add("test3");//not these usages var addresses = new List&lt;Address&gt;(); addresses.Add(new Address());//this usage only } } } </code></pre> <p>Find Usages will return all usages of <code>List&lt;T&gt;.Add</code> (which likely is very high in a real application)</p> <p>Search with pattern to the rescue again!</p> <p><img src="https://wallaceturner.azurewebsites.net/get/structural_search_and_replace3.png" alt="structural<em>search</em>and_replace3.png" /></p> <p>This returns only the one usage above instead of 4 usages using F12. </p> <p><strong>Important!</strong> Make sure when specifying the type of <code>$list$</code> that you specify the <em>fully qualified class name</em> eg <code>List&lt;ExampleApp.Person&gt;</code> - on that note it appears Resharper is showing the &lt; and > characters as their escaped versions in the UI which is a display-only thing.</p> <p><a href="http://wallaceturner.com/resharper-structural-search-and-replace-part-ii-adding-your-own-quick-fixes">Part II</a> will cover an example of finding usages and replacing them with a predefined template.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/resharper-structural-search-and-replace" /> 2012-02-20T11:02:00Z 2013-04-13T01:58:20Z Wallace Turner <p>There isn't much documentation on Resharper's <a href="http://blogs.jetbrains.com/dotnet/2010/04/introducing-resharper-50-structural-">Structural Search and Replace</a> (SSR) so here is 2 such ways you might find it useful.</p> <p>Consider the case where you want to find all the usages of the <code>Text</code> property on a Windows <code>Form</code>:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/form_usages.png" alt="form_usages.png" /></p> <p>If you use Find Usages (Shift F12) you get <em>all</em> usages of any inheritor of <code>System.Windows.Forms.Control</code> which isn't what we want in this case.</p> <p><img src="https://wallaceturner.azurewebsites.net/get/usages1.png" alt="usages1.png" /></p> <p>Enter Resharpers 'Search with pattern' (Alt-R,F,R)</p> <p><img src="https://wallaceturner.azurewebsites.net/get/structural_search_and_replace2.png" alt="structural<em>search</em>and_replace2.png" /></p> <p>which returns the following usages only:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/usages2.png" alt="usages2.png" /></p> <p>I can hear the guys up the back:</p> <blockquote> <p>you can already do that with Resharper's Advanced Find Usages (Ctrl+Shift+Alt+F12) </p> </blockquote> <p>but this only works in this instance because <code>Text</code> is overridden by <code>System.Windows.Form</code>. Consider a property of <code>Form</code> that isn't overridden (eg <code>form.Font</code>) - your only option here is to use SSR.</p> <h3>Other applications</h3> <p>You are searching for places where an <code>Address</code> is added to a <code>List&lt;Address&gt;</code> </p> <pre><code>namespace ExampleApp { public class Address { } public class Program { private static void SomeMethod() { var strings = new List&lt;string&gt;(); strings.Add("test1"); strings.Add("test2"); strings.Add("test3");//not these usages var addresses = new List&lt;Address&gt;(); addresses.Add(new Address());//this usage only } } } </code></pre> <p>Find Usages will return all usages of <code>List&lt;T&gt;.Add</code> (which likely is very high in a real application)</p> <p>Search with pattern to the rescue again!</p> <p><img src="https://wallaceturner.azurewebsites.net/get/structural_search_and_replace3.png" alt="structural<em>search</em>and_replace3.png" /></p> <p>This returns only the one usage above instead of 4 usages using F12. </p> <p><strong>Important!</strong> Make sure when specifying the type of <code>$list$</code> that you specify the <em>fully qualified class name</em> eg <code>List&lt;ExampleApp.Person&gt;</code> - on that note it appears Resharper is showing the &lt; and > characters as their escaped versions in the UI which is a display-only thing.</p> <p><a href="http://wallaceturner.com/resharper-structural-search-and-replace-part-ii-adding-your-own-quick-fixes">Part II</a> will cover an example of finding usages and replacing them with a predefined template.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/resharper-structural-search-and-replace" /> https://wallaceturner.azurewebsites.net/lzo-for-c C# Wrapper for 32 and 64 bit LZO <p><a href="http://www.oberhumer.com/opensource/lzo/">LZO</a> is a portable lossless data compression library written in ANSI C. It's <a href="http://www.oberhumer.com/opensource/lzo/#speed">fast</a>. Really <a href="http://lzo-net.sourceforge.net/documentation/benchmark.htm">fast</a></p> <p>I couldn't find a compiled 64-bit version for windows so I downloaded the source and compiled both the 32 bit and 64 bit versions:</p> <p><strong>Download now</strong> <a href="https://wallaceturner.azurewebsites.net/files/lzo2.dll">x86</a> <a href="https://wallaceturner.azurewebsites.net/files/lzo2_64.dll">x64</a></p> <p><a href="http://lzo-net.sourceforge.net/">lzo-net</a> is an existing sourceforge project that wraps the C LZO dll however <em>it is very outdated</em> - the lzo version in that project points at v1.08 and the latest is 2.06. That project is a single class <code>LZOCompressor</code> which handles the calls between your application and the C LZO library.*</p> <p>I added/fixed a few things as <code>LZOCompressor</code> no longer worked with v2.06 of the LZO C libraries:</p> <p>1) <code>DllImport</code> attributes needed fixing when calling from .NET 4<br /> 2) <a href="http://stackoverflow.com/questions/8241732/application-exits-no-exception-when-referencing-64bit-dll-from-c-sharp">String memory cleanup issue</a><br /> 3) Added feature: Detects 32bit or 64bit and loads the correct C LZO dll.</p> <h3>How to use in your .NET Application</h3> <p>A sample C# Application is <a href="https://wallaceturner.azurewebsites.net/files/Lzo64.zip">available to download</a> (838kb) which demonstrates using LZO from within either a 32bit or 64bit C# Application. This download is larger than you might expect because LZO C now <a href="http://www.dependencywalker.com/">depends</a> on the VC++ Redistributable library <code>msvcr100.dll</code> which I've included in the download.</p> <p>Happy compressing!</p> <p><em>*I've emailed the owner of this project but haven't heard back as yet</em></p> <img src="https://wallaceturner.azurewebsites.net/via-feed/lzo-for-c" /> 2011-11-30T11:58:20Z 2011-11-30T12:46:27Z Wallace Turner <p><a href="http://www.oberhumer.com/opensource/lzo/">LZO</a> is a portable lossless data compression library written in ANSI C. It's <a href="http://www.oberhumer.com/opensource/lzo/#speed">fast</a>. Really <a href="http://lzo-net.sourceforge.net/documentation/benchmark.htm">fast</a></p> <p>I couldn't find a compiled 64-bit version for windows so I downloaded the source and compiled both the 32 bit and 64 bit versions:</p> <p><strong>Download now</strong> <a href="https://wallaceturner.azurewebsites.net/files/lzo2.dll">x86</a> <a href="https://wallaceturner.azurewebsites.net/files/lzo2_64.dll">x64</a></p> <p><a href="http://lzo-net.sourceforge.net/">lzo-net</a> is an existing sourceforge project that wraps the C LZO dll however <em>it is very outdated</em> - the lzo version in that project points at v1.08 and the latest is 2.06. That project is a single class <code>LZOCompressor</code> which handles the calls between your application and the C LZO library.*</p> <p>I added/fixed a few things as <code>LZOCompressor</code> no longer worked with v2.06 of the LZO C libraries:</p> <p>1) <code>DllImport</code> attributes needed fixing when calling from .NET 4<br /> 2) <a href="http://stackoverflow.com/questions/8241732/application-exits-no-exception-when-referencing-64bit-dll-from-c-sharp">String memory cleanup issue</a><br /> 3) Added feature: Detects 32bit or 64bit and loads the correct C LZO dll.</p> <h3>How to use in your .NET Application</h3> <p>A sample C# Application is <a href="https://wallaceturner.azurewebsites.net/files/Lzo64.zip">available to download</a> (838kb) which demonstrates using LZO from within either a 32bit or 64bit C# Application. This download is larger than you might expect because LZO C now <a href="http://www.dependencywalker.com/">depends</a> on the VC++ Redistributable library <code>msvcr100.dll</code> which I've included in the download.</p> <p>Happy compressing!</p> <p><em>*I've emailed the owner of this project but haven't heard back as yet</em></p> <img src="https://wallaceturner.azurewebsites.net/via-feed/lzo-for-c" /> https://wallaceturner.azurewebsites.net/serialization-with-protobuf-net Serialization with protobuf-net <p><a href="http://code.google.com/p/protobuf-net/">protobuf-net</a> is an open source .net implementation of Google's <a href="http://en.wikipedia.org/wiki/Protocol_Buffers">protocol buffer</a> binary serialization format which I have used as a replacement for the <code>BinaryFormatter</code> serializer.</p> <h3>Setting it up</h3> <p>Consider the following classes, I have included a sub-class to make it slightly more than a trivial example:</p> <pre><code>public class Person { public string Name { get; set; } public int Age { get; set; } public DateTime DateOfBirth { get; set; } public Address Address { get; set; } } public class Address { public string Number { get; set; } public string StreetName { get; set; } } public class ExtendedAddress : Address { public string BuildingName { get; set; } } </code></pre> <p>With v2 of <code>protobuf-net</code> you can build a serializer on the fly which means you don't have to decorate the classes with attributes:</p> <pre><code>[SetUp] public void SetUp() { var protobufModel = ProtoBuf.Meta.TypeModel.Create(); AddTypeToModel&lt;Person&gt;(protobufModel); AddTypeToModel&lt;Address&gt;(protobufModel).AddSubType(500, typeof(ExtendedAddress)); AddTypeToModel&lt;ExtendedAddress&gt;(protobufModel); } private MetaType AddTypeToModel&lt;T&gt;(RuntimeTypeModel typeModel) { var properties = typeof(T).GetProperties().Select(p =&gt; p.Name).OrderBy(name =&gt; name);//OrderBy added, thanks MG return typeModel.Add(typeof(T), true).Add(properties.ToArray()); } </code></pre> <p>While this method is effective it is considered brittle if the class changes. What would happen if we added a <code>Lot</code> property to the <code>Address</code> class? This would break the backwards compatibility of any previously serialized data as <code>protobuf-net</code> is assigning ids to each serialized field. If we added a new property <code>Suburb</code> which is alphabetically after <code>StreetName</code> then backwards compatibility <em>would</em> be maintained albeit by sheer luck.</p> <p>The moral of the story is only use this method if your class isn't going to change (sure!) or you dont care about backwards compatibility.</p> <p>A more robust solution is to decorate your classes:</p> <pre><code>[ProtoContract] public class Person { [ProtoMember(1)] public string Name { get; set; } [ProtoMember(2)] public int Age { get; set; } [ProtoMember(3)] public DateTime DateOfBirth { get; set; } [ProtoMember(4)] public Address Address { get; set; } } [ProtoContract] [ProtoInclude(500, typeof(ExtendedAddress))] public class Address { [ProtoMember(1)] public string Number { get; set; } [ProtoMember(2)] public string StreetName { get; set; } } [ProtoContract] public class ExtendedAddress : Address { [ProtoMember(1)] public string BuildingName { get; set; } } </code></pre> <p><code>protobuf-net</code> also respects <code>DataContract</code> and <code>DataMember</code> attributes in the <code>System.Runtime.Serialization</code> namespace if you want to use them instead or if you class is attributed already.</p> <h3>Speed and size</h3> <p>Using <a href="http://autofixture.codeplex.com/">AutoFixture</a> I was able to create a <code>Person</code> object to test serialization with just a couple of lines of code that looks like this:</p> <p><img src="https://wallaceturner.azurewebsites.net/files/autofixture.png" alt="alt text" /></p> <p>The following is a summary of speed and size of the serialized data compared to the built-in <code>BinaryFormatter</code>. Tests conducted on an i7 870 @ 2.93GHz running Windows 7 on .Net4.0 in Release Mode using nunit-console. <link href="https://wallaceturner.azurewebsites.net/Content/site.css" rel="stylesheet" type="text/css"></p> <table class="sample"> <tr><td><td>Size (bytes)<td>Speed* (ms) <tr><td>BinaryFormatter<td>1024<td>3132 <tr><td>protobuf-net<td>256<td>458 </table> <p><em>*Serialization time is for 100,000 iterations. 1 iteration is a serialization followed by a deserialization</em></p> <p><code>protobuf-net</code> is a clear winner in both size (75% smaller) and speed (85% faster)</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/serialization-with-protobuf-net" /> 2011-11-03T07:00:00Z 2011-11-03T15:05:53Z Wallace Turner <p><a href="http://code.google.com/p/protobuf-net/">protobuf-net</a> is an open source .net implementation of Google's <a href="http://en.wikipedia.org/wiki/Protocol_Buffers">protocol buffer</a> binary serialization format which I have used as a replacement for the <code>BinaryFormatter</code> serializer.</p> <h3>Setting it up</h3> <p>Consider the following classes, I have included a sub-class to make it slightly more than a trivial example:</p> <pre><code>public class Person { public string Name { get; set; } public int Age { get; set; } public DateTime DateOfBirth { get; set; } public Address Address { get; set; } } public class Address { public string Number { get; set; } public string StreetName { get; set; } } public class ExtendedAddress : Address { public string BuildingName { get; set; } } </code></pre> <p>With v2 of <code>protobuf-net</code> you can build a serializer on the fly which means you don't have to decorate the classes with attributes:</p> <pre><code>[SetUp] public void SetUp() { var protobufModel = ProtoBuf.Meta.TypeModel.Create(); AddTypeToModel&lt;Person&gt;(protobufModel); AddTypeToModel&lt;Address&gt;(protobufModel).AddSubType(500, typeof(ExtendedAddress)); AddTypeToModel&lt;ExtendedAddress&gt;(protobufModel); } private MetaType AddTypeToModel&lt;T&gt;(RuntimeTypeModel typeModel) { var properties = typeof(T).GetProperties().Select(p =&gt; p.Name).OrderBy(name =&gt; name);//OrderBy added, thanks MG return typeModel.Add(typeof(T), true).Add(properties.ToArray()); } </code></pre> <p>While this method is effective it is considered brittle if the class changes. What would happen if we added a <code>Lot</code> property to the <code>Address</code> class? This would break the backwards compatibility of any previously serialized data as <code>protobuf-net</code> is assigning ids to each serialized field. If we added a new property <code>Suburb</code> which is alphabetically after <code>StreetName</code> then backwards compatibility <em>would</em> be maintained albeit by sheer luck.</p> <p>The moral of the story is only use this method if your class isn't going to change (sure!) or you dont care about backwards compatibility.</p> <p>A more robust solution is to decorate your classes:</p> <pre><code>[ProtoContract] public class Person { [ProtoMember(1)] public string Name { get; set; } [ProtoMember(2)] public int Age { get; set; } [ProtoMember(3)] public DateTime DateOfBirth { get; set; } [ProtoMember(4)] public Address Address { get; set; } } [ProtoContract] [ProtoInclude(500, typeof(ExtendedAddress))] public class Address { [ProtoMember(1)] public string Number { get; set; } [ProtoMember(2)] public string StreetName { get; set; } } [ProtoContract] public class ExtendedAddress : Address { [ProtoMember(1)] public string BuildingName { get; set; } } </code></pre> <p><code>protobuf-net</code> also respects <code>DataContract</code> and <code>DataMember</code> attributes in the <code>System.Runtime.Serialization</code> namespace if you want to use them instead or if you class is attributed already.</p> <h3>Speed and size</h3> <p>Using <a href="http://autofixture.codeplex.com/">AutoFixture</a> I was able to create a <code>Person</code> object to test serialization with just a couple of lines of code that looks like this:</p> <p><img src="https://wallaceturner.azurewebsites.net/files/autofixture.png" alt="alt text" /></p> <p>The following is a summary of speed and size of the serialized data compared to the built-in <code>BinaryFormatter</code>. Tests conducted on an i7 870 @ 2.93GHz running Windows 7 on .Net4.0 in Release Mode using nunit-console. <link href="https://wallaceturner.azurewebsites.net/Content/site.css" rel="stylesheet" type="text/css"></p> <table class="sample"> <tr><td><td>Size (bytes)<td>Speed* (ms) <tr><td>BinaryFormatter<td>1024<td>3132 <tr><td>protobuf-net<td>256<td>458 </table> <p><em>*Serialization time is for 100,000 iterations. 1 iteration is a serialization followed by a deserialization</em></p> <p><code>protobuf-net</code> is a clear winner in both size (75% smaller) and speed (85% faster)</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/serialization-with-protobuf-net" /> https://wallaceturner.azurewebsites.net/rename-property-to-existing-property-name-using-resharper Renaming duplicate properties using Resharper <p>You might come across some code like this</p> <pre><code>public bool IsOpen { get; set; } public bool Open { get; set; } </code></pre> <p>Don't question the <em>why</em> - just accept its evolved and requires cleaning up.</p> <p>Assume both are used in many places (otherwise you can do a manual refactor easily)</p> <p>If you use Resharper to Rename (Ctrl-R,R) <code>IsOpen</code> to <code>Open</code> Resharper will only replace the calls in that class and external calls wont be changed (ie it doesnt work)</p> <p>The trick is to <em>comment out</em> the property you want to keep:</p> <pre><code>//public bool IsOpen { get; set; } public bool Open { get; set; } </code></pre> <p>Now you can Rename <code>Open</code> and Resharper will do its magic.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/rename-property-to-existing-property-name-using-resharper" /> 2011-09-19T07:00:00Z 2011-09-19T11:33:13Z Wallace Turner <p>You might come across some code like this</p> <pre><code>public bool IsOpen { get; set; } public bool Open { get; set; } </code></pre> <p>Don't question the <em>why</em> - just accept its evolved and requires cleaning up.</p> <p>Assume both are used in many places (otherwise you can do a manual refactor easily)</p> <p>If you use Resharper to Rename (Ctrl-R,R) <code>IsOpen</code> to <code>Open</code> Resharper will only replace the calls in that class and external calls wont be changed (ie it doesnt work)</p> <p>The trick is to <em>comment out</em> the property you want to keep:</p> <pre><code>//public bool IsOpen { get; set; } public bool Open { get; set; } </code></pre> <p>Now you can Rename <code>Open</code> and Resharper will do its magic.</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/rename-property-to-existing-property-name-using-resharper" /> https://wallaceturner.azurewebsites.net/debugging-dump-files-created-on-another-machine Debugging dump files created on another machine <p><strong><em>Disclaimer</strong> - None of this article describes anything new or ground breaking, indeed Visual Studio 2010 has been out for quite some time. Developers don't usually spend their whole day debugging dumps until such time that there is a problem - thus this article serves as a quick refresher in times of need.</em></p> <h3>Intro</h3> <p>Visual Studio 2010 can open .Net4 process dump files to present you with a snapshot of the process in a familiar environment. </p> <p>This is handy when machines running your application can't easily be debugged (eg a client or production machine)</p> <p>Creating a dump file using Task Manager is easy (from Vista onwards) - simply right click the process and choose create dump. </p> <p><strong>However</strong>, If you want to take the .dmp file and analyse it on your own PC you need to ensure you have taken the correct dump. (<em>assuming you want to do Managed Debugging in Visual Studio 2010, I'm not talking about WinDBG</em>)</p> <p>Consider the matrix below: </p> <p><style type="text/css"> table.sample { border-width: 1px; border-spacing: 2px; background-color: white; font-family : Helvetica,Verdana,,Arial,sans-serif; empty-cells:hide; text-align:center; border-collapse: separate; } table.sample th { border-width: 1px; padding: 1px; border-style: inset; border-color: gray; background-color: white; -moz-border-radius: ; } table.sample td { border-width: 1px; padding: 15px; border-style: inset; border-color: gray; }</p> <p></style></p> <table class="sample"> <tr><td></td> <td></td><td align="center" colspan="2" bgcolor="lightgray">OS architecture</td></tr> <tr><td></td><td></td><td>32 bit</td><td>64 bit </td></tr> <tr><td rowspan="2" bgcolor="lightgray">Process running as</td> <td>32 bit</td><td bgcolor="lightgreen">Task Manager</td> <td bgcolor="yellow">Task Manager (32 bit)<br>C:\Windows\SysWOW64\taskmgr.exe</td> <tr><td>64 bit</td><td>N/A</td><td bgcolor="lightgreen">Task Manager</td></tr> </tr> </table> <h3>Getting the correct dump</h3> <p>If you are running a 32bit OS and developing an app that targets x86 but deploying to a 64bit server then the yellow box is your scenario - <s>you need to install Debugging Tools (x86) on the remote machine.</s> <strong>UPDATE</strong> You can use the 32 bit version of Task Manager (located at <code>C:\Windows\SysWOW64\taskmgr.exe</code>) to create the dump file; this is <em>much</em> easier.</p> <p>If you still want to use <a href="http://download.microsoft.com/download/A/6/A/A6AC035D-DA3F-4F0C-ADA4-37C8E5D34E3D/setup/WinSDKDebuggingTools/dbg_x86.msi">Debugging Tools x86</a>, view a <a href="http://wallaceturner.com/debugging-dump-files-created-on-another-machine?revision=46">previous version</a> of this article which describes how to take the dump.</p> <h3>Opening the dump file</h3> <p>The easiest way is to first open your .sln for the program you are debugging then drag the .dmp file onto VS <em>*or</em> use File -> Open. Suppose your production code was compiled under C:\buildserver\myapp and you are running the solution from D:\development\myapp - VS is smart enough to find the source <em>provided you opened the solution prior</em></p> <p>You'll see something like:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/vs_minidump_summary.jpg" alt="vs<em>minidump</em>summary.jpg" /></p> <p>If 'Debug with Mixed' is <a href="http://www.greyorgray.com/">greyed</a> out then your process probably isn't a .Net4 one (check CLR Versions)</p> <h3>Debug with Mixed</h3> <p>After clicking 'Debug with Mixed' you might see this:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/vs_start_debugging_error.jpg" alt="vs<em>startdebugging</em>error.jpg" /></p> <p>I'll paste the text of the error message so as to assist someone searching for this error message. </p> <blockquote> <p>Unable to find or download required files for managed minidump debugging. See Output window for additional information. Managed minidump debugging is disabled. Restart debug session after trying one or more of the following steps</p> </blockquote> <p>The output window had lots of stuff but this bit was interesting </p> <blockquote> <p>Managed Minidump Debugging: The debugger was unable to find or download version 4.00.30319.237 of 'mscordbi.dll'.</p> </blockquote> <p>Searching for this dll under C:\Windows\Microsoft.NET I find my version is indeed different, namely 4.00.30319.235.<br /> Running Windows Update <strong>fixed the issue</strong> by bringing my framework version up to date. At the time, the current version was 4.00.30319.237. Visual Studio did not download the symbols because I had unticked the Public Symbols from the debug options (see below). After ticking this and restarting 'Debug with Mixed' there was a long wait while VS downloaded the symbols. Make sure you also tick 'All modules' at least the first time so as to get all the missing pdbs for that version of the framework. <strong>You can uncheck this afterwards as it slows the loading of the dump file</strong></p> <p><img src="https://wallaceturner.azurewebsites.net/get/debug_options.png" alt="debug_options.png" /></p> <h3>No Source Available</h3> <p>The next issue you might face is this one: </p> <p><img src="https://wallaceturner.azurewebsites.net/get/no_source_available.png" alt="no<em>source</em>available.png" /></p> <p>When you get this the first thing to check is the Modules Window (Ctrl-D,M) and examine the Symbol Load Information for the exe/dll in question.</p> <p><img src="https://wallaceturner.azurewebsites.net/get/modules.png" alt="modules.png" /></p> <p>VS had a pretty good look for them as well:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/symbol_load_information.png" alt="symbol<em>load</em>information.png" /></p> <p>Place your pdbs and dlls in the same folder as the dump file. Are the dlls really required? I have been able to debug some dumps without them but since they sit alongside the pdbs its easy to not think about it and just copy them too. For more info on that see <a href="http://msdn.microsoft.com/en-us/library/htzy3t6f.aspx">this</a> article which states</p> <blockquote> <p>When you debug a dump file, the computer on which you debug must have access to the PDB symbol files and the binaries for the program. Visual Studio can cope with missing binaries for some modules, but it must have binaries for enough modules to generate valid call stacks. Otherwise, the "No matching binary found" message appears in the Modules window. </p> </blockquote> <p>If for some reason VS isnt loading your modules from the dump folder then define a directory under <code>Tools -&gt; Options -&gt; Debugging -&gt; Symbols</code> and put your pdbs/dlls in there. I have seen this occur but not in a consistent fashion.</p> <p>If the pdb's are the wrong ones you'll see this in the Symbol Load Information:</p> <blockquote> <p>D:\temp\DebuggingExampleApp.pdb: PDB does not match image.</p> </blockquote> <p>At this stage you need to pray you do have the correct pdb somewhere.</p> <h3>Finally</h3> <p>After you 'Debug With Mixed' the debugger should switch to Thread 1 and the relevant line in the source code automatically.</p> <p>If you know of any other quirks/issues let me know - usually your dmp file holds all the answers and the last thing you want is to be held up by roadblocks (or forced to use WinDBG :P)</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/debugging-dump-files-created-on-another-machine" /> 2011-08-30T02:12:00Z 2011-12-23T03:41:20Z Wallace Turner <p><strong><em>Disclaimer</strong> - None of this article describes anything new or ground breaking, indeed Visual Studio 2010 has been out for quite some time. Developers don't usually spend their whole day debugging dumps until such time that there is a problem - thus this article serves as a quick refresher in times of need.</em></p> <h3>Intro</h3> <p>Visual Studio 2010 can open .Net4 process dump files to present you with a snapshot of the process in a familiar environment. </p> <p>This is handy when machines running your application can't easily be debugged (eg a client or production machine)</p> <p>Creating a dump file using Task Manager is easy (from Vista onwards) - simply right click the process and choose create dump. </p> <p><strong>However</strong>, If you want to take the .dmp file and analyse it on your own PC you need to ensure you have taken the correct dump. (<em>assuming you want to do Managed Debugging in Visual Studio 2010, I'm not talking about WinDBG</em>)</p> <p>Consider the matrix below: </p> <p><style type="text/css"> table.sample { border-width: 1px; border-spacing: 2px; background-color: white; font-family : Helvetica,Verdana,,Arial,sans-serif; empty-cells:hide; text-align:center; border-collapse: separate; } table.sample th { border-width: 1px; padding: 1px; border-style: inset; border-color: gray; background-color: white; -moz-border-radius: ; } table.sample td { border-width: 1px; padding: 15px; border-style: inset; border-color: gray; }</p> <p></style></p> <table class="sample"> <tr><td></td> <td></td><td align="center" colspan="2" bgcolor="lightgray">OS architecture</td></tr> <tr><td></td><td></td><td>32 bit</td><td>64 bit </td></tr> <tr><td rowspan="2" bgcolor="lightgray">Process running as</td> <td>32 bit</td><td bgcolor="lightgreen">Task Manager</td> <td bgcolor="yellow">Task Manager (32 bit)<br>C:\Windows\SysWOW64\taskmgr.exe</td> <tr><td>64 bit</td><td>N/A</td><td bgcolor="lightgreen">Task Manager</td></tr> </tr> </table> <h3>Getting the correct dump</h3> <p>If you are running a 32bit OS and developing an app that targets x86 but deploying to a 64bit server then the yellow box is your scenario - <s>you need to install Debugging Tools (x86) on the remote machine.</s> <strong>UPDATE</strong> You can use the 32 bit version of Task Manager (located at <code>C:\Windows\SysWOW64\taskmgr.exe</code>) to create the dump file; this is <em>much</em> easier.</p> <p>If you still want to use <a href="http://download.microsoft.com/download/A/6/A/A6AC035D-DA3F-4F0C-ADA4-37C8E5D34E3D/setup/WinSDKDebuggingTools/dbg_x86.msi">Debugging Tools x86</a>, view a <a href="http://wallaceturner.com/debugging-dump-files-created-on-another-machine?revision=46">previous version</a> of this article which describes how to take the dump.</p> <h3>Opening the dump file</h3> <p>The easiest way is to first open your .sln for the program you are debugging then drag the .dmp file onto VS <em>*or</em> use File -> Open. Suppose your production code was compiled under C:\buildserver\myapp and you are running the solution from D:\development\myapp - VS is smart enough to find the source <em>provided you opened the solution prior</em></p> <p>You'll see something like:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/vs_minidump_summary.jpg" alt="vs<em>minidump</em>summary.jpg" /></p> <p>If 'Debug with Mixed' is <a href="http://www.greyorgray.com/">greyed</a> out then your process probably isn't a .Net4 one (check CLR Versions)</p> <h3>Debug with Mixed</h3> <p>After clicking 'Debug with Mixed' you might see this:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/vs_start_debugging_error.jpg" alt="vs<em>startdebugging</em>error.jpg" /></p> <p>I'll paste the text of the error message so as to assist someone searching for this error message. </p> <blockquote> <p>Unable to find or download required files for managed minidump debugging. See Output window for additional information. Managed minidump debugging is disabled. Restart debug session after trying one or more of the following steps</p> </blockquote> <p>The output window had lots of stuff but this bit was interesting </p> <blockquote> <p>Managed Minidump Debugging: The debugger was unable to find or download version 4.00.30319.237 of 'mscordbi.dll'.</p> </blockquote> <p>Searching for this dll under C:\Windows\Microsoft.NET I find my version is indeed different, namely 4.00.30319.235.<br /> Running Windows Update <strong>fixed the issue</strong> by bringing my framework version up to date. At the time, the current version was 4.00.30319.237. Visual Studio did not download the symbols because I had unticked the Public Symbols from the debug options (see below). After ticking this and restarting 'Debug with Mixed' there was a long wait while VS downloaded the symbols. Make sure you also tick 'All modules' at least the first time so as to get all the missing pdbs for that version of the framework. <strong>You can uncheck this afterwards as it slows the loading of the dump file</strong></p> <p><img src="https://wallaceturner.azurewebsites.net/get/debug_options.png" alt="debug_options.png" /></p> <h3>No Source Available</h3> <p>The next issue you might face is this one: </p> <p><img src="https://wallaceturner.azurewebsites.net/get/no_source_available.png" alt="no<em>source</em>available.png" /></p> <p>When you get this the first thing to check is the Modules Window (Ctrl-D,M) and examine the Symbol Load Information for the exe/dll in question.</p> <p><img src="https://wallaceturner.azurewebsites.net/get/modules.png" alt="modules.png" /></p> <p>VS had a pretty good look for them as well:</p> <p><img src="https://wallaceturner.azurewebsites.net/get/symbol_load_information.png" alt="symbol<em>load</em>information.png" /></p> <p>Place your pdbs and dlls in the same folder as the dump file. Are the dlls really required? I have been able to debug some dumps without them but since they sit alongside the pdbs its easy to not think about it and just copy them too. For more info on that see <a href="http://msdn.microsoft.com/en-us/library/htzy3t6f.aspx">this</a> article which states</p> <blockquote> <p>When you debug a dump file, the computer on which you debug must have access to the PDB symbol files and the binaries for the program. Visual Studio can cope with missing binaries for some modules, but it must have binaries for enough modules to generate valid call stacks. Otherwise, the "No matching binary found" message appears in the Modules window. </p> </blockquote> <p>If for some reason VS isnt loading your modules from the dump folder then define a directory under <code>Tools -&gt; Options -&gt; Debugging -&gt; Symbols</code> and put your pdbs/dlls in there. I have seen this occur but not in a consistent fashion.</p> <p>If the pdb's are the wrong ones you'll see this in the Symbol Load Information:</p> <blockquote> <p>D:\temp\DebuggingExampleApp.pdb: PDB does not match image.</p> </blockquote> <p>At this stage you need to pray you do have the correct pdb somewhere.</p> <h3>Finally</h3> <p>After you 'Debug With Mixed' the debugger should switch to Thread 1 and the relevant line in the source code automatically.</p> <p>If you know of any other quirks/issues let me know - usually your dmp file holds all the answers and the last thing you want is to be held up by roadblocks (or forced to use WinDBG :P)</p> <img src="https://wallaceturner.azurewebsites.net/via-feed/debugging-dump-files-created-on-another-machine" />