Released LINQ to Twitter v5.1.0

Today, I released LINQ to Twitter v5.1.0. This is a .1 release to fix bugs and add a few features that are new in the Twitter API. I’ll explain what’s new in this post. In the meantime, if you just want to grab the latest version of LINQ to Twitter, here are the links:

Removed Deprecated Methods

Generally, taking away from APIs is a breaking change, so I don’t do it lightly. I regularly mark members with an ObsoleteAttribute for several releases before removal. In all of the cases below, the endpoints have been removed from the Twitter API and you can’t use them anyway:

  • UpdateAccountBackgroundImageAsync
  • All DirectMessage APIs (replaced by DirectMessageEvents API)
  • Control, Site, and User Streams

Twitter pages haven’t had a background image for a while – thus removing this made sense. The DirectMessage APIs have been replaced with DirectMessageEvent APIs and are no longer available.

Control, Site, and User streams are no longer part of the Twitter API.

Added performBlock Parameter to ReportSpamAsync

Previously, you could just report spammers with ReportSpamAsync. This new performBlock parameter lets you also block a user at the same time as reporting, like this:

User spammer = await twitterCtx.ReportSpamAsync(
    SpammerScreenName, performBlock: true);

Added AdditionalOwners Parameter to UploadMediaAsync

According to the Twitter API docs, AdditionalOwners is:

A comma-separated list of user IDs to set as additional owners allowed to use the returned media_id in Tweets or Cards. Up to 100 additional owners may be specified.

The media_id that Twitter refers to is the MediaID property of the Media instance returned by the call to UploadMediaAsync, like this:

byte[] imageBytes = File.ReadAllBytes(@"..\..\..\images\TwitterTest.mp4");
const ulong JoeMayoUserID = 15411837;
var additionalOwners = new ulong[] { JoeMayoUserID };
string mediaType = "video/mp4";
string mediaCategory = "tweet_video";

Media media = await twitterCtx.UploadMediaAsync(
    imageBytes, mediaType, additionalOwners, mediaCategory);

The additionalOwners parameter is an IEnumerable<ulong>, belonging to an optional overload of UploadMediaAsync.

Added URL and Description Entities to User Object

On a user’s Twitter page, they can include a URL and Description. As with tweets, the Twitter API can extract URLs so you don’t have to do the parsing yourself. In addition to the profile URL, the Twitter API also extracts any URLs from a user’s profile description.

This LINQ to Twitter update adds an Entities property to the User object that makes it easy for you to read these URL entities. Here’s an example of a UserType.Show query:

var user =
    await
    (from usr in twitterCtx.User
     where usr.Type == UserType.Show &&
           usr.ScreenName == screenName &&
           usr.TweetMode == TweetMode.Extended
     select usr)
    .SingleOrDefaultAsync();

By setting the TweetMode parameter to TweetMode.Extended in the query, LINQ to Twitter will populate the Entities property on the user instance.

Added TweetMode Parameter to TweetAsync and ReplyAsync

This change adds a new parameter, TweetMode, to existing overloads because it makes sense to always be an option.

Back when Twitter started offering 280 character tweets, they represented this as Extended mode. The default was and still is Compatibility mode, which is the previous 140 character limit. To maintain consistency with the Twitter API behavior and documentation, LINQ to Twitter follows the same default convention, like this:

Status tweet = 
    await twitterCtx.TweetAsync(
        status, tweetMode: TweetMode.Extended);

Without the tweetMode parameter, the tweet.Text property will be truncated to 140 characters. However, by setting tweetMode to the TweetMode.Extended enum, the tweet.Text property is null and tweet.FullText contains the user’s entire tweet, up to the max 280 characters.

Added parameters Parameter to BeginAuthorizationAsync

This update allows you to share state across queries during the OAuth process. When performing OAuth in web apps, there’s a multi-step process, which boils down to two steps in LINQ to Twitter; calling the BeginAuthorizationAsync and CompleteAuthorizationAsync methods over the Twitter API OAuth process.

Setting the parameters parameter, a Dictionary<string, string>, sends name/value query string parameters to the Twitter API, which then return after the user authorizes your application. You would put code an MVC Begin action method, like this:

var parameters = new Dictionary<string, string> 
{ 
    { "my_custom_param", "val" } 
    
};
string twitterCallbackUrl = 
    Request.GetDisplayUrl().Replace("Begin", "Complete");
return await auth.BeginAuthorizationAsync(
    new Uri(twitterCallbackUrl), parameters);

When the Twitter API OAuth process returns to the Close action method, you can read the MVC Request property to find the parameter values.

Added ExecuteRawAsync Overload That Accepts an HttpMethod Parameter

ExecuteRawAsync and RawQuery are features of LINQ to Twitter that help you implement Twitter API features that might not have made it into LINQ to Twitter yet. e.g. What if the Twitter API adds a new parameter to and endpoint that LINQ to Twitter doesn’t have yet.

In the early days of the Twitter API, most endpoints worked with either a GET or POST HTTP call. However, newer APIs are more specific. e.g. Operations that delete/remove accept a DELETE HTTP method. Previously, ExecuteRawAsync assumed a POST endpoint, but this new change gives you the ability to use any HTTP method, like this:

string result = 
    await twitterCtx.ExecuteRawAsync(
        queryString, parameters, HttpMethod.Post);

This uses the System.Net.HttpMethod enum that is part of a new ExecuteRawAsync method overload.

Final Thoughts

So, that’s it for this version. These changes cover most of the items that people have asked for or that I had on the issues list. There’s much more to do, so stay tuned and let me know what you think.

@JoeMayo

Recent/Upcoming Presentations

Last week, I was at Desert Code Camp, giving a “Deep Learning Basics” presentation. It was a good time speaking with other developers, sharing what I could, and answering questions.

If you haven’t heard anyone who speaks or teaches say this, the preparation makes you learn more about a subject than you might not have learned normally. The questions are also excellent. Some are similar from time to time (assuming it’s the same talk) but others are often windows into what people are thinking. e.g. just last week someone was interested in the retraining scenario where the data is constantly updating and you need to retrain the model. There isn’t any one answer and a couple people with different ideas participated in the discussion for a few minutes. While discussion veered a bit off topic for the subject, that was fun (hopefully for everyone).

Here’s the code and presentation slides:

Deep Learning Demos

In a little more than a week, Oct 26th, 2019, I’ll be speaking at the Phoenix Mobile Festival on “AI In Your App – ML.NET and Xamarin”. No code yet because this is the first time for this presentation and the bits aren’t ready. It’s been a while since I’ve worked with Xamarin (the mobile part), but I’m excited about ML.NET, which is a good option for C# developers to write AI code.

What’s great about these events is that they’re free. I don’t make any money and neither do organizers. I guess there could be a lot to be said about why people engage in such complex community building activities, but maybe it’s just a great experience. If you make it, please stop and say “Hi”.

@JoeMayo

Reading Tumbr Data

Tumblr is a site that has blogs, friends, followers, and all of the other things you expect in a social networking site. It also has a REST API that you can query.  Here’s an example that pulls blog information:

using System;
using System.Xml.Linq;

namespace Tumblr
{
    class Program
    {
        static void Main()
        {
            XElement tumblrResponse =
                XElement.Load("http://joemayo.tumblr.com/api/read");
            XElement posts = tumblrResponse.Element("posts");

            if (posts.Attribute("total").Value != "0")
            {
                foreach (var post in posts.Elements("post"))
                {
                    Console.WriteLine(
                        "nTitle: {0}nBody: {1}",
                        post.Element("regular-title").Value,
                        post.Element("regular-body").Value);
                }
            }

            Console.ReadKey();
        }
    }
}

You can see that I’m using LINQ to XML, pointing the XElement.Load at the URL on tumblr for reading. With LINQ to XML, I’m using the Element, Elements, and Attribute selectors to get at the information I need and display it.  The query processing is based on the following XML document returned from tumblr:

<tumblr version="1.0">
  <tumblelog name="joemayo" timezone="US/Eastern" title="Untitled"></tumblelog>
  <posts start="0" total="1">
    <post id="303295702"
             url=http://joemayo.tumblr.com/post/303295702
             url-with-slug=http://joemayo.tumblr.com/post/303295702/checking-out-tumblr
             type="regular" date-gmt="2009-12-27 22:27:57 GMT" date="Sun, 27 Dec 2009 17:27:57"
             unix-timestamp="1261952877" format="html" reblog-key="k5TElAb8" slug="checking-out-tumblr">
      <regular-title>Checking Out Tumblr</regular-title>
      <regular-body>&lt;p&gt;This is a test post to see how the Tumblr API works.&lt;/p&gt;</regular-body>
    </post>
  </posts>
</tumblr>

Joe

Free Weather Forecast Web Service

The National Weather Service has a Web service you can use:

http://www.weather.gov/xml/

To use it:

1. Copy the WSDL:

http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl

2. Right-click on the Web site and select Add Web Reference.

3. Paste the WSDL address, from step #1 above into the address bar and click Go.

4. After you see the response from the Web service, which is a description of the service, click the Add Reference button, which will add gov.weather.www to the Web References folder.

5. The following service shows one way to use the service:

        protected void Page_Load(object sender, EventArgs e)
        {
            var weather = new gov.weather.www.ndfdXML();
            var result = weather.LatLonListZipCode("80130");
            var denWeather =
                weather.GmlTimeSeries(
                    "39.5432,-104.939",
                    DateTime.Parse("12/31/09"),
                    DateTime.Parse("01/02/10"),
                    WebTest.gov.weather.www.compTypeType.Between,
                    WebTest.gov.weather.www.featureTypeType.Forecast_Gml2AllWx,
                    "maxt,mint,wx");
            Response.Write(denWeather);
        }

I called LatLonListZipCode to get the LatLon for the specified Zip Code.  Then I passed that LatLon to the first parameter of the call to GmlTimeSeries.  Yep, I cheated by hitting a break point and copying the latlon out of the XML document in result.  denWeather is another XML document that is returned by the service.  You can use LINQ to XML to get the return values and display them on the screen however you want.  You should visit http://www.weather.gov/xml/ for a full description of what methods are available, the meaning of their parameters, and any other information about the API.

What Open Source Development is Teaching Me

I’m learning a lot from the open-source project, LINQ to Twitter, that I started a little over a year ago. Surely not alone in thought, it was amazing why so many people spend their free time writing free software. After all, software development is a good business to be in and make a living by being paid well. Giving away hard work for free might seem crazy, but then again, it might make sense if viewed from other perspectives; such as technical challenge, showing off, and maybe a bit of altruistim.  In the following paragraphs, I’ll share some thoughts on my experience over the last year and we’ll see how far into insanity the journey has led.

Why do people climb mountains? Because they are there. Why would a developer want to build a CMS, and IDE, or even a LINQ Provider? It could be part necessity where you want to use the code for a specific purpose or maybe you just want to tackle a hard problem.  In my case, I was writing LINQ Programming and wanted to show the reader how to create their own LINQ Provider. Looking around, there were already many LINQ providers for all of the major databases and many other data sources, such as Amazon.com, Flikr, and more. One day in early 2008, I was reading Joe Stagner’s excellent blog and saw his Follow Me on Twitter badge. Out of curiosity, I checked it out and noticed that there was a Twitter API. I needed an idea for a LINQ Provider, Twitter had an API, and no one had written a LINQ provider for Twitter yet.  At the same time, CodePlex.com was really cranking up with great projects and offered Visual Studio Team System integration and I had been thinking about doing some open source work for a long time.  Actually, back in 2001, Christoph Wille had told me what a great experience he had with the SharpDevelop project, which is an open-source .NET IDE. It seems like the right set of circumstances were in place and I decided to release LINQ to Twitter as an open-source project on August 26th 2008.  While Microsoft has put important framework pieces in place and provided resources, writing a LINQ provider is no walk in the park. Like the mountain climber, the opportunity to challenge my skills beckons.

Another part of the challenge of an open source project is to show off.  If your work is good, then you might look like a capable developer, but it could easily backfire if your sloppy. I provide consulting and software development services for a living and it’s normal for customers to want to see my work. The problem with showing my work is that most of it is covered under non-disclosure, which is normal.  Therefore, I need something to prove that I can actually code and am not just a theorist who writes books. Open source provides something to show off with for the customer.  However, this is a double-edged sword.  Everyone has their own opinion on what is “good code’. For example, I know that there are idioms in my code, such as using var and m_ prefixes for private fields, that drive other developers batty.  What I decided was that there will always be a percentage of the population that will disagree with me, regardless of convention. That’s okay, because any time you put yourself out into the public eye, it’s only natural that everyone will not agree with you or like your work. Sometimes there are bugs both in my code and via Twitter, but my code is what people are using.  I remember at the beginning of 2009 when the Twitter API changed, requiring the HTTP Expect100Continue header to be set to false.  I hadn’t worked on the project for a while and didn’t notice until people started complaining. This taught me to keep watching for changes and be proactive in resolving problems instead of waiting for it to affect everyone using the software.  Generally, feedback has been very positive and the small amount of negative feedback has opened doors for new learning opportunities. Keeping the code in good shape and continuous improvement are essential and if you’re lucky it might be something that shows off well.

What goes around comes around might sound cliché, but it has normally led to positive experiences for me. There are a lot of people in the community who help others because they feel good doing something for other people. There are people in nearly every segment of society who selflessly pitch in and donate their time to good causes.  There have been a lot of people in my life to enabled me to get ahead, just out of the goodness of their heart. If I can do something to give back and show appreciation for that, then I’m happy to do so. All things in moderation, I have to work to make a living, but will probably always have a strong tendency to give to the community. So, writing open-source software through LINQ to Twitter is one of those avenues I’ve chosen for more community service. Other developers can look at the code and see how parts of C# can be used, such as interfaces, events, and various patterns.  Other developers wanting to write their own LINQ providers have another example to look at.  Of course, we can’t overlook the fact that there have been over 3,000 downloads in the last 12 months and growing of people who can actually use LINQ to Twitter to integrate Twitter into their own applications.  Does this make me feel good; Hell Yes!  As I said, many people do voluntarily provide some type of community service, which is excellent and I believe that people volunteering their time to the community in some way contributes to making this world a better place.

@JoeMayo

How Twitter can Help Developers – I

As a Twitter developer, I’ve had a fair amount of experience with their API and thought that I might express how I think Twitter could help developers.  Twitter interacts with the community, requesting feedback on new features, helping developers, and generally letting us know what is happening with API status at Twitter Development Talk.  Twitter also offers API Announce to let developers know what is happening with the API.  If you’ve been watching, the Twitter API team has been growing and evolving in a positive direction to help developers.  These are all good signs that indicate a continued dedicated effort toward improving their API, but there are still pain points that I’ll address in the following paragraphs, including testing, API availability, and roadmap.

One of the first things I noticed when coding against the Twitter API was the glaring absence of a test environment.  That means all coding and testing must be done against the production Twitter environment.  As a professional software engineer, this ripples against the grain of common sense and is still something that causes me hesitation any time I test.  The current answer is to set up a dummy account on Twitter to test with.  However, that still leaves me wondering about things like usage policies and the like.  For example, what if I wrote a test that slammed Twitter with tons of new requests.  Would an unfortunate result in my programming and testing behavior make me a hacker, a spammer, or just an idiot that didn’t desk-check his development code; and what would be the response from the folks at Twitter evaluating the results of my actions? To fix the problem with no test environment, Twitter should offer an endpoint for just testing and let developers establish test accounts.

The way Twitter is handling release of the latest API’s is to give a selected set of users access to a feature and a selected set of developers access to the associated API. This is normal with Beta testing and developers who weren’t included should normally not be concerned.  The problem occurs when the feature is given to the general public and the vast majority of developers are caught unprepared to support users.  Two features that follow this pattern are Lists and Retweets.  I’ll give Twitter credit for publishing specs, which at least gave some developers the ability to write code to the spec, but it doesn’t solve the problem.  The problem is that the situation can result in users frustrated because their 3rd party applications are slow in providing support for the new features. Just an observation, but those users could always dump their current 3rd party vendor and run to a 3rd party vendor that was included in Twitters beta program for early API access.  It would be helpful if Twitter released new APIs to the developer community before releasing the new Web site feature to the general public.

The Roadmap could be much better. The problems are that it is too tactical, lacks important information, and is useless for planning. If there were a good thing I could say about the roadmap is that it seems to be fairly well-organized and shows some of the items they have considered for implementation; although some items, such as the test environment plans discussed earlier, are nearly a year old.  Looking at the items on the list, most subject areas look like down-in-the weeds topics that are important in themselves, but provide no strategic direction or vision on where the API is headed. For example, one of the items is named “Return User object on end_session method”, which is important; but is important on only that part of the API. These tactical items are already on the bug list and Twitter could enhance the developer experience with a separate priority list for bugs; which might be entirely possible with the existing Web site tools.  Another problem is that the Roadmap doesn’t include information about upcoming APIs, which I believe is more appropriate content.  For example, what if the roadmap had included significant API features such as Retweets, Lists, or Geotracking, which are the type of things that should be included in a roadmap? The purpose of the roadmap should be for developers to look at what Twitter has planned and prepare accordingly. The Twitter roadmap has no information on priorities or planned implementation. If 3rd party applications want to plan for future growth, they can’t use the roadmap for any long term plans because Twitter publishes previously unannounced APIs before items on the roadmap are addressed.  The roadmap would be much more useful if it concentrated on strategic initiatives that included upcoming APIs with at least some indication of when the API will be implemented.

As Twitter continues to grow and 3rd party libraries and applications glom on, Twiiter should nurture the developer ecosystem they brought into being.  A testing platform is long overdue, proper release of APIs should be fair, and the roadmap should support planning and vision rather than being a hit list for heads-down coders.  I’m sure you might have a favorite item to put on Twitter’s developer ToDo list, but these are the things that I’m thinking about the most.

@JoeMayo

Solving AG_E_PARSER_BAD_TYPE In Silverlight

If you search for AG_E_PARSER_BAD_TYPE, you’ll find many entries going back to early Betas of Silverlight, which may or may not be applicable today. When I received this error, my search led me to Bill Kratochvil’s blog entry, AG_E_PARSER_BAD_TYPE Adding Silverlight Toolkit control to module.

In my particular problem, I was using Prism and my project with Shell.xaml (Project A) referenced another project module (Project B), which referenced two utility libraries (Project C and Project D). The occurred during the call to InitializeComponent() in a View in Project C. The line number and column in the exception message specified code in the View XAML that referred to an object in Project D.  Going back to Project A, where Shell.xaml resides, I observed that Project A referenced Project B and Project C, but not Project D.  I fixed the problem by adding a reference from Project A to Project D.

So, adding to Bill’s observation, it appears that this problem can be solved by locating the project of the object in the XAML where the error occurs (Project D) and ensuring that the project containing Shell.xaml (Project A) has a reference to Project D. 

EF Tip: Show in Designer

If you’ve ever opened the Visual Studio Entity Designer and spent more than a few seconds searching for a specific entity, you’ll quickly wonder if there must be a better way to do this.  Fortunately there is, through the Model Browser.  Finding the specific entity is only a few clicks away:

1. Open the Model browser.  You can select View > Other Windows > Model Browser:

image

2. When you see the Model Browser window, open the Conceptual Entity Model, which often has the “Model” suffix if you created the model with the default naming scheme.  In this example, I’m using the Northwind database, which is named NORTHWINDModel. Then open the Entity Types branch and you’ll see something like this:

image

3. To find the Customer entity, right-click on the Customer entity in the Model Browser and select Show in Designer, like this:

image

4. The designer automatically centers on your entity.

LINQ to Twitter Talks to Laconica

Got to looking at the Laconica API and noticed that they built a Twitter compatible API.  This got me thinking that LINQ to Twitter might work with Laconica.  So, I pointed LINQ to Twitter at http://jtweeter.com and fired it up:

var twitterCtx = new TwitterContext(userName, password, "http://jtweeter.com/api/", "http://search.twitter.com/");

var publicTweets =
    from tweet in twitterCtx.Status
    where tweet.Type == StatusType.Public
    select tweet;

publicTweets.ToList().ForEach(
    tweet => Console.WriteLine(
        "User Name: {0}, Tweet: {1}",
        tweet.User.Name,
        tweet.Text));

And Success!