Tuesday, June 13, 2017

Announcing Delphi Developer Days 2017 Special Edition Database Workshop

I am pleased to announce that registration is now open for Delphi Developer Days 2017 Special Edition Database Workshop with Bob Swart and me, Cary Jensen, which will be held this fall in the U.S. and Europe.

Unlike our traditional Delphi Developer Days events, which include a broad range of topics, this workshop focuses exclusively on database development. It begins with a detailed look at connecting to data and configuring FireDAC, and continues with a look at some of FireDAC’s more advanced features, including Array DML, the SQL command preprocessor, and cached updates.

The workshop continues with an in-depth discussion of multi-tier development. Topics include creating DataSnap servers and using Embarcadero’s RAD Server. Here you will learn how to access your data from mobile applications, how to support both Delphi and non-Delphi clients using REST (REpresentational State Transfer), and how to create RAD Server microservices with Delphi.

Both days of this workshop are taught jointly by me and Bob. You are encouraged (though not required) to bring a laptop in order to follow along with the provided code samples.

Each attendee will receive a Delphi Developer Days course book, which includes a special edition of my latest book Delphi in Depth: FireDAC, along with material on DataSnap, mobile development, and RAD Server written by Bob (Dr.Bob) Swart.

Here are the cities and dates for this fall tour:
Chicago, 25-26 September 2017
Antwerp, Belgium, 9-10 October 2017
Frankfurt, Germany, 27-28 November 2017

In addition to the course book, attendees also receive lunch on each day and all code samples covered during the workshop. Due to the nature of this workshop format, seating is limited. Register now to save your place in this special edition database workshop and receive a substantial discount for early registration. For more information, visit http://www.DelphiDeveloperDays.com.

The Delphi Developer Days 2017 Special Edition Database Workshop is taught in English.

Note that we will return with the traditional Delphi Developer Days event in the spring of 2018, where we present both joint as well as separate sessions on a wide variety of Delphi topics.


Friday, May 12, 2017

Delphi in Depth: FireDAC is now available

Last autumn, as I was finishing up my chapters for the 2016 Delphi Developer Days course book, which I wrote with Nick Hodges, Director of Engineering at Embarcadero Technologies, I received a question about FireDAC from my German colleague Dr. Holger Flick. He was asking about the relevance of my ClientDataSet book to some work he was doing with FireDAC. I replied that the book would help to some extent since so many aspects of Delphi database development were covered by that book, and that I had used FireDAC and InterBase for many of that book’s examples.

But his question got me to thinking about the FireDAC material that I had already written. I had four or five chapters from previous Delphi Developer Days, I had another three or four articles that I had published in software-related magazines, and I had several FireDAC presentations from CodeRage under my belt. I might have half a book there already. Why not, I thought, my next book will be a FireDAC book.

I don’t regret that decision at all, though I must say that even I was surprised at how much work it was to complete this book. This is my twenty-sixth book (and that’s not counting the eight Delphi Developer Days books that we’ve published since 2009). I know how much work it takes to write a book, but this one was different. The amount of research that this book required was a surprise.

For one thing, I felt that this book should be more than just a FireDAC book. It should be a book that deserves to be on the shelf of every Delphi database developer. For the new Delphi database developer who is unfamiliar with the “Delphi way” of doing things, this book contains discussions of and demonstrations of the essential techniques that every Delphi database developer needs to know. This includes concepts like the TDataSet interface, the current record, and how to read to and write from TFields. (Just a note here, this book assumes you are using Delphi XE6 or later.)

The advanced Delphi database developer, however, is not left out. These developers may simply want to scan some of the chapters where core concepts are reviewed, but there are many chapters that delve into some of the more powerful and exciting features of FireDAC, including virtual field management, cloned cursors, and cached updates.

So, the short story is that the book was a lot of work, and in the end, worth the effort.
The book begins with an overview of FireDAC, including its features and benefits. In chapter 2 you learn the many different ways to connect to your data, whether you are using one of the mainstream databases and a FireDAC native driver or are migrating from a legacy file-server database.

Chapter 3 explores how you configure FireDAC to control nearly every aspect of your data retrieval, in-memory management, and subsequent database writes. Here you learn about FireDAC dataset property inheritance, and how to fine tune configurations at the individual component level.

Chapters 4 and 5 demonstrate how you communicate with the underlying database using dataset components, queries, and stored procedures. Here I describe the capabilities of these components, as well as some of their basic configuration, including how to create and use parameterized query and stored procedure calls. I also cover how to use transactions, as well as how to monitor the database connections from your running applications.

Chapter 6 introduces the fundamentals of Delphi database development, introducing concepts like the current record, data-bound navigation and editing, as well as programmatic navigation and editing. Chapter 7 introduces the creation and use of indexes, while chapters 8 and 9 discuss searches and filtering, many techniques of which employee indexes.

These four chapters are specifically intended to provide a core understanding of data-related operations in Delphi. I expect the more advanced Delphi database developers to quickly scan chapter 6 for anything that doesn’t look familiar. However, even the advanced developer may glean new and useful information from a more thorough read of chapters 7 through 9, as several of the technical reviewers noted (and these guys are definitely advanced Delphi database developers).

The remaining chapters of this book are dedicated to more advanced FireDAC technologies that can enable sophisticated solutions in your database applications. Chapter 10 introduces virtual fields, and shows you how to use aggregates, Calculated (and InternalCalc) fields, and Lookup fields. It also examines the FieldsOptions property, a feature introduced in Delphi XE6.

In chapter 11 I cover how to persist your FireDAC datasets and FDSchemaAdapters to files and streams, as well as how to retrieve that data. There’s a lot of good stuff in that chapter, including how to control what data is persisted, and how data being loaded can interact with data that is already loaded.

In Chapter 12 and 13 I explore the FDMemTable. Here I discuss the role played by the FDMemTable within the FireDAC landscape, and then examine many of the techniques that are best suited for FDMemTables. Topics include the dynamic creation of FDMemTable structure, loading FDMemTables from other datasets, cloning cursors, and creating nested datasets.

In Chapter 14 I take an in-depth look at the SQL command preprocessor. Here you learn how to add tremendous flexibility to your SQL statements, features that permit you to run a common set of SQL statements across two or more different database servers. The SQL command preprocessor also gives you unprecedented control over expressions that you add to your SQL, introducing dozens of FireDAC scalar functions for your use.

Chapter 15 covers Array DML (Data Manipulation Language), a high-speed mechanism for executing a large number of similar parameterized queries, for instance, when you are inserting thousands of records into a given table with an INSERT SQL statement. When you need to update or insert a large amount of data quickly, this is your tool.

In Chapter 16 I examine a feature that I have been a champion of for years, cached updates. Here you will learn how to create, manage, and exploit cached updates to create advanced features for your applications. I also discuss both the decentralized and centralized models of cached updates, and provide important guidance for the process of applying updates and handling potential update failures.

I finish the book with a short chapter on Local SQL, demonstrating how to configure your FireDAC components to perform SQL queries on any Delphi dataset or combination of datasets. It’s a powerful technique that provides for solutions that would previously be difficult or impossible.

This has been a challenging while fun and rewarding project for me, and for those of you who buy my book, thank you! I hope you enjoy it.

Still interested? Learn more here: http://www.jensendatasystems.com/FireDACbook

Monday, November 7, 2016

A Special Year for Delphi Developer Days

Delphi Developer Days 2016 with Nick Hodges and me, Cary Jensen, is just around the corner (we start in Chicago a week from today). And, the Delphi Developer Days book has been printed, which for an author is that one milestone that is always rewarding. As in years past, this book is once again weighing in at about 400 pages.

This year is very special for another reason. We started planning Delphi Developer Days 2016 back in late May, and this year I have the pleasure of working with my longtime friend and former Delphi Product Manager Nick Hodges.

Nick has been a prolific writer over the past couple of years, publishing not one, but two Delphi books (Coding in Delphi and More Coding in Delphi). I was really looking forward to working with him on the Delphi Developer Days book, and I was not let down one bit. This book is loaded with good stuff, and I cannot wait for our chance to present this material.

But that is not the special part I am talking about. My past Delphi Developer Days presenters have all been serious contributors to the Delphi community. Marco Cantù has gone on to become the current Delphi Product Manager. Bob Swart continues to speak at conferences, publish articles, and runs the Delphi Developer Network in the Netherlands (and I look forward to his return to Delphi Developer Days as my co-presenter next year). Ray Konopka continues his great work, and both Konopka Signature Components and Code Site are bundled with various versions of Delphi. And then, in September, we learned that Nick had joined Embarcadero as the Director of Product Management, and will be overseeing Delphi as well as other Embarcadero products.

But even with that great responsibility, Nick is committed to completing this autumn's Delphi Developer Days tour. And what a special gift for our attendees. Not only are they going to hear one of the really gifted Delphi presenters, but they will be interacting with one of the people Embarcadero has entrusted with shepherding Delphi, and other products, into the future.

There are still some seats available for our Chicago and Baltimore events, and Frankfurt is almost full (Copenhagen has been sold out for over a month now). So, if you want to be a part of one of the coolest Delphi events this year, you have got to act soon. Oh, yeah, and you get your own copy of this year's Delphi Developer Days 2016 book. (Sorry, this book is available only to Delphi Developer Days attendees.)


Tuesday, October 11, 2016

When Migrating to FireDAC, You Must Understand UpdateMode

Let's start with a little history lesson. When Delphi first shipped it had one data access framework, the Borland Database Engine (BDE). While the BDE was a breakthrough product in its early years, providing a fast, independent, and easy-to-use data access layer, it was cumbersome to install, used a lot of network bandwidth, and had limited support for remote database servers. Over time, it became increasingly obsolete.

The need for a new data access mechanism for Delphi became even more apparent during the development of Kylix, a Delphi-based compiler and IDE (integrated development environment) for Linux. Porting the BDE to Linux was ruled out, and dbExpress was born. The dbExpress data access framework is high-speed and client/server friendly, largely based on pass-through SQL (structured query language).

The dbExpress framework has one major drawback, however. In most cases converting a BDE project to dbExpress required a major refactoring of the data access logic, and dbExpress did not support the old-style file server databases, such as Paradox, dBase, or MS Access until Delphi XE2. As a result, for most many developers dbExpress was a poor option.

FireDAC changes that. Conversion from the BDE to FireDAC is pretty smooth, and Delphi even ships with a tool, named reFind, that helps with much of the conversion process. It is for this reason that I use FireDAC in all of my new projects (and all of the database projects included in my source code for Delphi Developer Days). In addition, I think that a good argument can be made for migrating legacy applications to use FireDAC when a major revision is scheduled. Yes, it is that good.

One Big Difference: UpdateOptions.UpdateMode

One reason why migration to FireDAC from the BDE is so easy is that FireDAC provides a solid implementation of the TDataSet Interface, and all of the basic operations available in the BDE can be found in FireDAC. Better still, FireDAC implements many of the higher-end features previously only found in ClientDataSets. As a result, if you know the BDE and/or ClientDataSets, using FireDAC is a no brainer.

There is one very significant difference between how FireDAC and the BDE/ClientDataSets work, and this difference is very easy to miss, in part because it is related to the default value of a single property. And because the effects of this property have such a profound impact on how your database applications work, it is important that you know about this difference when you decide to start using FireDAC. The property is UpdateMode.

UpdateMode is a property found in the UpdateOptions property of FireDAC TDataSets. It is also a property of BDE TDataSets as well as the TDataSetProvider when used to apply updates to a caching ClientDataSet. In all of these mentioned classes the UpdateMode property performs the same task, it influences how the corresponding component generates the SQL used to write changes to the underlying database.

The UpdateMode property is of the type TUpdateMode. The following is the declaration of the TUpdateMode type:

TUpdateMode = (upWhereAll, upWhereChanged, upWhereKeyOnly);

Here is how it works. When UpdateMode is set to upWhereAll, all non-BLOB fields are included in the WHERE clause of UPDATE and DELETE queries. This results in update failures if any of the non-BLOB fields of the underlying record were modified since the time that the table was opened, the query or stored procedure was executed, or the FDMemTable was loaded. This approach is known as optimistic locking, and when two or more users are trying to apply changes at about the same time to the same record, only the first to apply changes will succeed. All others will fail.

When UpdateMode is set to upWhereChanged, only the primary field keys and the fields that have been modified are included in the WHERE clause of UPDATE queries. (Again, INSERTs are not affected. DELETE queries continue to use an exact match criteria since there are no changed fields in a deleted record.) As long as none of the primary key fields of an updated record are affected, and all non-primary key fields that have been modified have also not been updated in the underlying table since the time the data was loaded into the DataSet, these queries should succeed.

Using upWhereChanged permits two or more users to apply their changes to the underlying database so long as none of them have made changes to common fields. For example, if one user changed a field called Address, and another changed a field called PhoneNumber, and neither of these fields are part of the primary key, both users will successfully apply their changes. This type of update permits merges.

The final UpdateMode value is upWhereKeyOnly. With UpdateMode set to upWhereKeyOnly, the WHERE clause of UPDATE queries only includes the values of the primary key fields. (INSERT and DELETE queries continue to act as they do with upWhereChanged.) Using this mode, so long as the key fields of the underlying record have not been changed, the updates are applied, replacing any updates that other users may have applied.

Assuming that key fields are not touched (this is a pretty safe assumption in most database architectures), the use of upWhereKeyOnly permits everyone to succeed in posting their changes. As a result, the last user to post is the user whose data appears in the underlying database.

The Difference and Why It Is Important

The default value of FireDAC's UpdateOptions.UpdateMode is upWhereKeyOnly, and this is a really big deal. It means that if you use FireDAC datasets without changing this default value, there is a good chance that some users may report that their changes have disappeared. Or, even worse, never know that their edits have been overwritten.

Here is how this can happen. If two or more users read the same record (by way of a query, stored procedure call, or by opening an FDTable), and two or more post a change to that record, the last user to post their record will replace those changes posted before them. What's problematic about this is that the users who posted before the last user will have no idea that their changes have been overwritten.

By comparison, most developers prefer to use either pessimistic locking (the first user to start editing the record prevents any other user from editing the record until changes have been posted), or optimistic locking (once the first user posts a change to a record, subsequent attempts to post to that same record will fail, since the original record no longer can be found, based on the WHERE clause predicates). In these scenarios, the first user to post wins, and other users must first re-read the record, after which they can decide whether or not to update the newly posted contents.

FireDAC defaults to an UpdateMode of upWhereKeyOnly, since the queries required to update the database tend to execute faster. It is up to you, however, to decide whether or not the performance improvement is more important that the possible loss of data.

The DataSetProvider class, the class which ClientDataSets use to resolve changes back to the underlying database, and BDE DataSets (TTable, TQuery, and TStoredProc) also have an UpdateMode property. For these objects, the default value of UpdateMode is upWhereAll, the conservative setting that prevents a user from overwriting another user's edits.

So, the bottom line is this. You need to understand how FireDAC's UpdateOptions.UpdateMode affects how records are updated in the underlying database, and set this property to the value that meets the needs of your application.

An Easy Solution, If You Want to Change UpdateMode

Fortunately, another feature of FireDAC makes it easy to change the default value of UpdateOptions.UpdateMode property. The UpdateOptions property, like a number of other properties in FireDAC, is found in a number of classes in the FireDAC framework, including FDManager, FDConnection, FDQuery, FDMemTable, and so on.

These classes are related in a hierarchical sense, in that the settings found in the UpdateOptions property of the FDManager (a singleton object) are inherited by all FDConnections configured to use the FDManager. Similarly, any FireDAC dataset, such as an FDQuery or FDMemTable, will employ the settings found in the FDConnection to which they are connected.

So, if your application is using one FDConnection, changing the UpdateOptions.UpdateMode property of that FDConnection will automatically apply that setting to any FDTable, FDQuery, FDStoredProc using that connection (so long as you have not already touched the UpdateOptions.UpdateMode property on any of these TDataSets). If you are using more than one FDConnection, add an FDManager and change the value of the UpdateOptions.UpdateMode property there, and that change will likewise cascade down to any FDConnections, and then on down to the individual TDataSets, again, so long as that property has not also been overridden in any of the individual FDConnections or TDataSets.

Note that if your FDQuery, FDStoredProc, or FDTable is already connected to an FDConnection, and that FDConnection currently uses upWhereKeyOnly, changing the FDConnection UpdateOptions.UpdateMode property alone might not change the FireDAC DataSet. You might have to disconnected, and then re-connect the DataSet to the FDConnection before the DataSet's UpdateMode property updates.

Copyright © 2016 Cary Jensen. All Right Reserved

Tuesday, July 12, 2016

Delphi Developer Days 2016 with Nick Hodges and Cary Jensen Agenda Published

The agenda and course topics for this year's Delphi Developer Days with Nick Hodges and me, Cary Jensen, has been released, and we think that you will be pleased with the selection of topics. Over the course of this two-day event we will have a combination of talks, some of which are jointly presented by Nick and me, as well as talks where we break out into separate rooms to cover a variety of topics.

For our joint sessions we are covering threading and thread synchronization, the parallel programming library, code de-coupling, dependency injection, and a debugging deep dive. We are also going to discuss best practices in professional development, a thought-provoking look at the principles and philosophies crucial to success in the art of software development. And finally, no Delphi Developer Days tour would be complete without the Tip, Tricks, and Techniques session, were Nick and I share some of our favorite tricks-of-the-trade.

I am really excited about presenting these joint sessions with Nick. Over the past couple of years Nick and I have co-presented talks at several Delphi conferences, and each time it was not only fun for us, but also provided our audiences with a deeper understanding of the subject matter. Nick and I are passionate about the topics we picked for our joint sessions, and the opportunity to present them jointly gives us the chance to really explore the issues, and provide a contrast of opinions and solutions.

Topics for our separate talks including unit testing, FireDAC, advanced Delphi language features, patterns in Delphi, building Windows services, and using JSON (JavaScript Object Notation). As in the past, these joint sessions are concurrent, meaning that you'll have to choose between two simultaneous topics. But you'll miss nothing, as Nick and I are publishing the material for each session in the Delphi Developer Days course book. This course book, which in the past has been well over three hundred pages long, is only available to attendees of Delphi Developer Days.

Along the way we'll also have a keynote address from Embarcadero, question and answer sessions, and the opportunity to network with fellow Delphi developers. For a detailed agenda and descriptions of each of the topics, please visit http://www.DelphiDeveloperDays.com/descriptions.html.

We are very fortunate to have Nick join us for Delphi Developer Days this year. Over the past several years Nick has published two books on Delphi, Coding in Delphi and More Coding in Delphi. He is getting ready to publish his next book, Dependency Injection in Delphi, and you don't want to miss this chance to hear one of the leading Delphi experts share his views and insights. I hope to see you at Delphi Developer Days 2016 in Chicago, Copenhagen, Frankfurt, or Baltimore.

Register by August 12, 2016 and save 25% off of the regular price. For more information, and to register, visit http://www.DelphiDeveloperDays.com

Tuesday, June 7, 2016

Delphi Developer Days 2016 with Nick Hodges and Cary Jensen

It is my great pleasure to announce that Nick Hodges will be joining me for this year's Delphi Developer Days tour, which will visit the United States, Europe, and Scandinavia late this autumn. Delphi Developer Days are intense, two-day training events focusing on Delphi, the cross-platform, rapid application development platform from Embarcadero Technologies.

The tour will begin in Chicago on November 14th and 15th, and will continue in Copenhagen, Denmark on November 24th and 25th. We will be in Frankfurt, Germany on November 28th and 29th, and will conclude the tour in the Baltimore/Washington DC area on December 5th and 6th.

Delphi Developer Days consists of both joint talks, where Nick and I present a topic together, as well as individual sessions where we focus on various aspects of Delphi development. As we have done in the past, each attendee will receive a printed book containing the material from all of our talks. In the past these books have been in excess of 350 pages long, and this year looks to be no different.

I am especially proud to have Nick Hodges presenting with me this year. Nick is one of the original members of the Delphi community, and the author of arguably the first custom Delphi component (TSmiley). He is the recipient of the coveted Spirit of Delphi award, one of Embarcadero's Delphi MVPs, and a regular presenter at Delphi conferences and web casts. Nick previously served as the Delphi Product Manager, and has recently published two celebrated books on Delphi (Coding in Delphi, and More Coding in Delphi).

Nick is both an excellent presenter and writer, two talents that are essential for Delphi Developer Days. He is the perfect addition to the list of co-presenters that I've had the honor to work with over the years. My Delphi Developer Days co-presenters in previous years were Marco Cantú (currently the RAD Studio Product Manager), Bob (Dr.Bob) Swart, and Ray Konopka (Raize Software). Bob is still involved with Delphi Developer Days, but is unable to commit to travel this fall. I look forward to his return to Delphi Developer Days in 2017, but in the meantime, I am delighted to have Nick stepping in as a first-rate replacement.

Nick and I are currently working on our agenda, and we will announce our topics in the next couple of weeks. Registration is now open at www.DelphiDeveloperDays.com, and if you register in the next several weeks (prior to August 12, 2016), you will receive a 25% discount on the registration fee. We've sold out many of our events in the past, and with Nick on board for this tour, I anticipate that seats will go fast.

Just a final note. Normally Delphi Developer Days tours occur in the spring. We had to postpone our spring tour this year due to the unexpected loss of a close family member. Thank you to all who supported us during this difficult time, and thank you also for your patience. We are very happy that we are able to provide a 2016 tour, and with the addition of Nick, it's guaranteed to be one of the best Delphi Developer Days tours yet.

Sunday, January 24, 2016

Delphi Developer Days Postponed

The annual spring Delphi seminar known as Delphi Developer Days has been postponed due to circumstances beyond our control.

Loy Anderson (the DDD event manager), Bob (Dr.Bob) Swart, and I have been working to put together this year's Delphi Developer Days, and we were getting close to announcing our dates when Loy and I suffered the tragic and sudden loss of a close family member. The circumstances of this event are going to require that Loy and I spend a significant amount of time helping out our family. Unfortunately, it is not possible to provide this help and at the same time invest the time required to make Delphi Developer Days a success.

We will make an announcement once we have re-scheduled the Delphi Developer Days dates. If you would like to be informed once new dates have been chosen, you can visit www.DelphiDeveloperDays.com and click the link labeled Notify Me. I will also blog and tweet once we know when the next DDD event will be offered, as well as post this information to the official Delphi Developer Days site and the Delphi Developer Days Facebook page (https://www.facebook.com/delphideveloperdays).

Thank you for your understanding, and we look forward to seeing you at the next Delphi Developer Days...

Cary