Building an RSS reader for Android #4: Data access objects and SQL queries

Picking up where I previously left off, a DAO is an abstract class or interface that contains definitions of methods for interacting with our database. Collectively the methods can be described as CRUD—Create, Read, Update, and Delete, which together comprise the standard four operations of a persistent storage system. Furthermore we don’t actually implement the methods ourselves; the Room library does this for us, although depending on the complexity of a given operation, some SQL (structured query language) is necessary.

To keep things organized, I created a few separate DAOs, three of them corresponding to the three tables or entities we defined previously (Feed, Entry, and FeedEntryCrossRef). For example, FeedsDao contains all operations that have to do with Feeds:

interface FeedsDao {

    @Insert(onConflict = OnConflictStrategy.IGNORE)
    fun addFeeds(vararg feed: Feed)

    @Query("SELECT * FROM Feed WHERE url = :feedId")
    fun getFeed(feedId: String): LiveData<Feed?>

    @Query("SELECT url, title, imageUrl, category, unreadCount FROM Feed")
    fun getFeedsLight(): LiveData<List<FeedLight>>

    @Query("SELECT url, title, website, imageUrl, description, category FROM Feed")
    fun getFeedsManageable(): LiveData<List<FeedManageable>>

    @Query("SELECT url FROM Feed")
    fun getFeedIds(): LiveData<List<String>>

    @Query("SELECT url, category FROM Feed")
    fun getFeedIdsWithCategories(): LiveData<List<FeedIdWithCategory>>

    @Query("SELECT url FROM Feed")
    fun getFeedUrlsSynchronously(): List<String>

    @Query("SELECT title FROM Feed WHERE url = :feedId")
    fun getFeedTitleSynchronously(feedId: String): String

    @Update
    fun updateFeed(feed: Feed)

    @Query("UPDATE Feed SET title = :title WHERE url = :feedId")
    fun updateFeedTitle(feedId: String, title: String)

    @Query("UPDATE Feed SET category = :category WHERE url IN (:feedId)")
    fun updateFeedCategory(vararg feedId: String, category: String)

    @Transaction
    fun updateFeedTitleAndCategory(feedId: String, title: String, category: String) {
        updateFeedTitle(feedId, title)
        updateFeedCategory(feedId, category = category)
    }

    @Query("UPDATE Feed SET imageUrl = :feedImage WHERE url = :feedId")
    fun updateFeedImage(feedId: String, feedImage: String)

    @Query("UPDATE Feed SET unreadCount = :count WHERE url = :feedId")
    fun updateFeedUnreadCount(feedId: String, count: Int)

    @Query("UPDATE Feed SET unreadCount = (unreadCount + :addend) WHERE url = :feedId")
    fun incrementFeedUnreadCount(feedId: String, addend: Int)

    @Query(
        "UPDATE Feed SET unreadCount = (unreadCount + :addend) WHERE url IN " +
            "(SELECT url FROM FeedEntryCrossRef AS _junction " +
            "INNER JOIN Feed ON (_junction.feedUrl = Feed.url) " +
            "WHERE _junction.entryUrl = (:entryId))"
    )
    fun incrementFeedUnreadCountByEntry(entryId: String, addend: Int)

    @Query("DELETE FROM Feed WHERE url IN (:feedId)")
    fun deleteFeeds(vararg feedId: String)
}

How an operation gets defined within this interface is determined by a few things: 1) what specific data we want retrieved, if any, 2) what data we have on hand by which to fetch something else (for example, primary keys, or any other property), and 3) whether the data should be fetched synchronously or asynchronously—I’ll come back to this point later.

Simple operations

The simplest operations are those that don’t return any data. For instance, addFeeds accepts a variable number of arguments (vararg), that is, any number of Feeds that we want added to our database. We simply annotate it with @Insert and Room itself creates the necessary database query. We also include OnConflictStrategy.IGNORE to tell Room to ignore any incoming Feed whose primary key (in this case, its URL) already exists in our database.

@Insert(onConflict = OnConflictStrategy.IGNORE)
fun addFeeds(vararg feed: Feed)

Another simple operation is updateFeed, which takes as an argument a single Feed. By simply annotating it with @Update, Room knows to use the Feed’s primary key to query the database and make the necessary changes.

@Update
fun updateFeed(feed: Feed)

SQL queries, and synchronous vs asynchronous operations

In most other cases, we need to write the SQL queries ourselves. SQL is a standard way of communicating with relational databases. For example, the method getFeed, which takes a single feedId (or its URL) as an argument, is annotated with @Query, into which we pass a simple query as a string: SELECT * FROM Feed WHERE url = :feedId. This means we want to select all (*) columns from all rows in our Feed table whose value of url is equal to the feedId that has been passed into the method. Because we’ve ensured that all Feed URLs in our table are unique, we expect to find only one Feed.

@Query("SELECT * FROM Feed WHERE url = :feedId")
fun getFeed(feedId: String): LiveData<Feed?>

Aside: the above method returns to us a nullable Feed wrapped in a LiveData object. Most of the methods defined in this database are asynchronous; this is because database transactions require a perceivable amount time to complete, much like an HTTP request. So as to not make our user interface wait so much, a LiveData object is returned instantly without its content. This LiveData is observable by our UI; as soon as the transaction completes, its content is updated automatically.

There are indeed cases where we want to fetch data synchronously, such as in background tasks where the user interface is not involved. Here we simply skip using LiveData:

@Query("SELECT url FROM Feed")
fun getFeedUrlsSynchronously(): List<String>

@Query("SELECT title FROM Feed WHERE url = :feedId")
fun getFeedTitleSynchronously(feedId: String): String

Because database transactions are expensive, in many cases we need to define our database operations so that we only fetch precisely the data that we need and nothing more. In the above two methods, for example, each one returns only one field from the Feed table. For cases where we need multiple fields, I created different variations of our Feed data class, each with a different set of properties depending on our needs.

// Light version of Feed – no website and description
data class FeedLight(
    val url: String,
    var title: String,
    val imageUrl: String?,
    var category: String,
    var unreadCount: Int
)

// Feed without unreadCount
data class FeedManageable(
    val url: String,
    var title: String,
    val website: String,
    val imageUrl: String?,
    val description: String?,
    var category: String
): Serializable

data class FeedIdWithCategory(
    val url: String,
    val category: String
)

The following methods show how the above data classes come into play. In our queries, we specify which fields we want in particular, and Room does the job of mapping them onto our specified return objects. In the case of getFeedIds, we only care about one field, url, which is a String. Note also that each of these methods returns a list of objects (wrapped in LiveData), exactly what we expect from a query that ends with FROM Feed without qualifiers—meaning it will fetch data from all rows of our Feed table.

@Query("SELECT url, title, imageUrl, category, unreadCount FROM Feed")
fun getFeedsLight(): LiveData<List<FeedLight>>

@Query("SELECT url, title, website, imageUrl, description, category FROM Feed")
fun getFeedsManageable(): LiveData<List<FeedManageable>>

@Query("SELECT url FROM Feed")
fun getFeedIds(): LiveData<List<String>>

@Query("SELECT url, category FROM Feed")
fun getFeedIdsWithCategories(): LiveData<List<FeedIdWithCategory>>

Making queries with limited data

Earlier I mentioned updateFeed, which is an operation made simple for us to define and annotate because it accepts an entire Feed as an argument. There’s no need to write an explicit SQL query because Room has all the information it needs from a complete Feed object. But there are many cases in which we might want to communicate with our database by passing in only a limited amount of data.

For example, updateFeedTitle accepts only a feedId (or URL) and a new title as arguments, so we include a query: UPDATE Feed SET title = :title WHERE url = :feedId. This means it will update a Feed’s title column with the given title, in any row where the Feed’s url column is the same as the given feedId.

@Query("UPDATE Feed SET title = :title WHERE url = :feedId")
fun updateFeedTitle(feedId: String, title: String)

The method updateFeedCategory is similar, except it accepts a variable quantity of feedId (URLs) and exactly one category. This means that it will update a Feed’s category column with the given category, in each row in which the feedId is found among the given collection of feedId’s.

@Query("UPDATE Feed SET category = :category WHERE url IN (:feedId)")
fun updateFeedCategory(vararg feedId: String, category: String)

Combined methods

Methods may also be combined into a single transaction with the @Transaction annotation, in cases where we want them executed at the same time.

@Transaction
fun updateFeedTitleAndCategory(feedId: String, title: String, category: String) {
    updateFeedTitle(feedId, title)
    updateFeedCategory(feedId, category = category)
}

Complex queries

The method incrementFeedUnreadCount, as its name suggests, is meant to increment a Feed’s unreadCount property by any given number, which could be positive or negative. It simply takes a feedId as an argument, along with a desired addend, and finds the desired Feed to which the given feedId belongs.

@Query("UPDATE Feed SET unreadCount = (unreadCount + :addend) WHERE url = :feedId")
fun incrementFeedUnreadCount(feedId: String, addend: Int)

But what if we wanted to perform the same operation by passing in an entryId instead of a feedId? Remember that Feed and Entry have a many-to-many relationship; this means we want the operation to apply to all Feeds that are associated with any given Entry. In such a case, we need to consider related data across different tables, which requires a more complex query.

So, we annotate the method incrementFeedUnreadCountByEntry with a nested query: UPDATE Feed SET unreadCount = (unreadCount + :addend) WHERE url IN (SELECT url FROM FeedEntryCrossRef AS _junction INNER JOIN Feed ON (_junction.feedUrl = Feed.url) WHERE _junction.entryUrl = (:entryId)).

First, let’s look at the inner query: this time, we select a Feed’s url property from our FeedEntryCrossRef table, using it as a junction (and, appropriately, giving it the variable name _junction) between our Feed and Entry entities. The INNER JOIN clause allows us to join the Feed table to our FeedEntryCrossRef table, by establishing an equivalence between Feed.url and _junction.feedUrl. In the resulting joined table, the query selects a Feed’s url field from all rows in which the value of entryUrl is equivalent to the given entryId, giving us a collection of one or more Feed URLs.

From here the outer query is made much simpler: it looks for all rows of Feed in which the value of url is present in the above collection, and adds the given addend (which, again, may be positive or negative) to the existing value of unreadCount.

@Query(
        "UPDATE Feed SET unreadCount = (unreadCount + :addend) WHERE url IN " +
            "(SELECT url FROM FeedEntryCrossRef AS _junction " +
            "INNER JOIN Feed ON (_junction.feedUrl = Feed.url) " +
            "WHERE _junction.entryUrl = (:entryId))"
    )
fun incrementFeedUnreadCountByEntry(entryId: String, addend: Int)

All of the above represent database operations that are relevant only to Feeds, though in one instance we have needed to consider our FeedEntryCrossRef table. As I mentioned before, there are other DAOs in the app—I’ll write about them next time.

Building an RSS Reader for Android #3: Setting up a relational database, many-to-many relationships, and Room

Previously in this series, I wrote about FeedParser and FeedSearcher, each representing a data source for the app. The former is responsible for fetching and parsing RSS feeds, while the latter is responsible for fetching a collection of information about RSS feeds that can be subscribed to. To clarify, they are not true data sources, since they don’t store or provide data themselves; each one is simply a layer meant to access data from somewhere else, although the rest of the app has no knowledge of this. In both cases, the actual data source is remote: for FeedParser, raw XML pages from the web; and for FeedSearcher, Feedly’s search engine.

So we have a way of fetching basic information about an RSS feed (its URL, title, description, etc.), and a way to to actually retrieve the contents of the feed itself, including a collection of entries. But to actually enable subscription to a feed, the app needs a third, local data source that it can use to access previously parsed feeds—otherwise, to read a feed that we’ve already subscribed to, we would have to keep fetching and parsing it from the internet. This is where a local database comes in.

Room, entities, and many-to-many relationships

Room is Android’s standard library for local data persistence. According to the documentation, it “provides an abstraction layer over SQLite to allow fluent database access while harnessing the full power of SQLite.” This greatly reduces the amount of work required for database access, as opposed to using the SQLIte APIs directly, which is more error-prone and involves significant boilerplate code. (Read the documentation for more information.)

In my first post, I hinted that NiceFeed is concerned mainly with two kinds of data objects: Feed and Entry. We define these as data classes, specifying all the relevant properties, and annotating each of them with @Entity. This tells Room that each of these data classes represents a table. Additionally, for each entity we specify a @PrimaryKey, signifying that the annotated property is an item’s unique identifier; for Feed and Entry, each one’s primary key is its URL.

@Entity
data class Feed(
    @PrimaryKey val url: String, // Doubles as Feed ID
    var title: String,
    val website: String,
    val description: String? = null,
    val imageUrl: String? = null,
    var category: String = "Uncategorized",
    var unreadCount: Int
) : Serializable

@Entity
data class Entry(
    @PrimaryKey val url: String, // Doubles as Entry ID
    val title: String,
    val website: String,
    val author: String?,
    val date: Date?,
    val content: String?,
    val image: String?,
    var isStarred: Boolean = false,
    var isRead: Boolean = false
) : Serializable {

    ...
}

I also mentioned before that the relationship between Feed and Entry is many-to-many. This simply means that a Feed can be associated with multiple entries, which is usually the case. Occasionally, a single Entry will be part of more than one Feed. For example, an article from the New York Times may be included both in the Latest News and Travel feeds. This is important to note so that we don’t end up with duplicate articles in our database.

To create an association between one Feed and one Entry, we define a third data class and entity: FeedEntryCrossRef, which simply contains the URLs of one Feed and one Entry. This entity represents a junction by which two other tables can be joined; it tells us that that the Feed whose URL is stored here is associated with the Entry whose URL is also stored here. (Check out this tutorial on SQLite joins for more information.)

@Entity(
    primaryKeys = ["feedUrl", "entryUrl"],
    indices = [(Index(value = ["entryUrl"]))]
)
data class FeedEntryCrossRef(
    val feedUrl: String,
    val entryUrl: String
)

Defining the database

Now to create our database. We define NiceFeedDatabase as an abstract class which extends Room’s predefined RoomDatabase; this is because Room itself handles the building. Within the @Database annotation for Room, we pass in our three entities—Feed, Entry, and FeedEntryCrossRef—and mark it as version 1. And within the companion object, we define a static method for actually building the database, so that we can call on the class directly upon starting the app to initialize our database.

@Database(
    entities = [
        Feed::class,
        Entry::class,
        FeedEntryCrossRef::class
    ],
    version = 1
)
@TypeConverters(com.joshuacerdenia.android.nicefeed.data.local.database.TypeConverters::class)
abstract class NiceFeedDatabase : RoomDatabase() {

    abstract fun combinedDao(): CombinedDao

    companion object {

        private const val DATABASE_NAME = "database"

        fun build(context: Context): NiceFeedDatabase {
            return Room.databaseBuilder(
                context.applicationContext,
                NiceFeedDatabase::class.java,
                DATABASE_NAME
            ).build()
        }
    }
}

Type converters

Additionally, notice that there’s a second annotation, @TypeConverters. This is because SQLite only directly handles basic data types like strings and integers. But Room provides a way to easily convert others into types that SQLite will handle. Our Entry data class includes a Date, representing the date it was published. So we define a TypeConverters class to contain methods that tell Room what to do when it receives a Date to store into our database (fromDate), and when we ask for one (toDate). Then we pass this class into our @TypeConverters annotation above.

class TypeConverters {

    @TypeConverter
    fun fromDate(date: Date?): Long? {
        return date?.time
    }

    @TypeConverter
    fun toDate(millisSinceEpoch: Long?): Date? {
        return millisSinceEpoch?.let {
            Date(it)
        }
    }
}

Data access objects—to be continued

And with that, our database setup is nearly complete. The one final thing to note is the abstract function that I have named combinedDao. This represents our data access objects (or DAO, for short)—basically, an interface or intermediately layer through which our app can access data from our database. Within combinedDao (so named because it’s a combination of multiple smaller interfaces, each having to do with one of our three entities), I have defined all methods necessary for the app to get the data that it needs in various cases. NiceFeed is a mildly complex app, so I ended up needing a significant number of these methods, some involving complex SQL queries. I’ll write about this in a future post.

Building an RSS Reader for Android from nothing #2: Searching for feeds using Feedly’s Search API

Previously, I wrote about the component that handles retrieving and parsing raw RSS feeds from the web. This component represents one of NiceFeed’s main data sources, giving us an actual RSS feed, in the form of interrelated model objects that we’ve defined (Feed, Entry, FeedWithEntries), which the app can then display. All we need to do is give it the URL of any particular RSS feed, and it does all the work.

In real life, nobody’s going to want to keep entering URLs manually; sometimes, a user won’t even know what they are. A respectable RSS reader would have to have some way for the user to search for new feeds. Therefore, we need another data source—this data source would accept a keyword and return to the user a list of RSS feeds from the internet based on that keyword. The user would then able to choose any of these RSS feeds to subscribe to.

Feedly’s public Search API exists for this very purpose. It accepts a string search query as input, with some optional parameters such as the number of desired search results and a specific locale; in return, we get an organized collection of data in the form of JSON. This data, basically a collection of RSS feeds, is organized into fields such as “title,” “website,” “feedId,” “description,” and so on.

To get this data into the app, we pick the fields we want and create a new data class containing relevant properties corresponding to those fields. I named this data class SearchResultItem, which represents one item (containing data from one RSS feed) in a list of search results.

data class SearchResultItem(
    val title: String?,
    @SerializedName("feedId") val id: String?,
    val website: String?,
    val description: String?,
    val updated: String?,
    @SerializedName("visualUrl") val imageUrl: String?
): Serializable

To actually accomplish the task of communicating with Feedly API to obtain the needed data, some housekeeping is needed. I used the Retrofit library to do all the heavy lifting with regard to networking and parsing JSON from Feedly, making it very easy to implement the whole thing. All that’s needed is a simple interface which sends the request to the API, with an annotation that defines the request—in this case @GET, which represents the method for fetching data. The method returns a Call, which represents one HTTP request, with the parameter SearchResult—a class defined solely to contain one or more SearchResultItems—representing the type of a successful response.

interface FeedlyApi {
    @GET
    fun fetchSearchResult(@Url url: String): Call<SearchResult>
}
class SearchResult {
    @SerializedName("results")
    lateinit var items: List<SearchResultItem>
}

And now for the main event. I created a class called FeedSearcher to contain all the methods needed to carry out a Feedly search request. It has one public method: getFeedList, which takes a String query or keyword, and returns a LiveData wrapping a list of SearchResultItems.

/*  Generates a search query and returns a list of results from Feedly */
class FeedSearcher(private val networkMonitor: NetworkMonitor) {

    private val retrofit = Retrofit.Builder()
        .baseUrl(BASE_URL)
        .addConverterFactory(GsonConverterFactory.create())
        .build()
    private val feedlyApi = retrofit.create(FeedlyApi::class.java)

    fun getFeedList(query: String): LiveData<List<SearchResultItem>> {
        return if (networkMonitor.isOnline) {
            val path = generatePath(query)
            val request: Call<SearchResult> = feedlyApi.fetchSearchResult(path)
            fetchSearchResult(request)
        } else {
            MutableLiveData(emptyList())
        }
    }

    private fun generatePath(query: String): String {
        return Uri.Builder()
            .path("v3/search/feeds")
            .appendQueryParameter("count", RESULTS_COUNT.toString())
            .appendQueryParameter("query", URLEncoder.encode(query, "UTF-8"))
            .build()
            .toString()
    }

    private fun fetchSearchResult(
        request: Call<SearchResult>
    ): MutableLiveData<List<SearchResultItem>> {
        val searchResultLiveData = MutableLiveData<List<SearchResultItem>>()
        val callback = object : Callback<SearchResult> {
            override fun onFailure(call: Call<SearchResult>, t: Throwable) {} // Do nothing

            override fun onResponse(
                call: Call<SearchResult>,
                response: Response<SearchResult>
            ) {
                val feedSearchResult = response.body()
                searchResultLiveData.value = feedSearchResult?.items ?: emptyList()
            }
        }

        request.enqueue(callback)
        return searchResultLiveData
    }

    companion object {
        private const val RESULTS_COUNT = 100
        private const val BASE_URL = "https://cloud.feedly.com/"
    }
}

There are several things going on here. First we need a Retrofit object with a base URL of “https://cloud.feedly.com/” to be able to do anything—it accepts the above FeedlyApi interface, and uses it to create a request. To the request itself, we pass a String path to make a complete search URL on top of the base URL; the private method generatePath accepts our String query and returns it in the form (which includes a specified number of search results, in this case 100) that the Feedly search engine expects.

Finally, within the private method fetchSearchResult, the request object is enqueued, which actually executes the HTTP request. The method immediately returns a LiveData to the previous method that called it. We also pass in a callback object that defines what should happen after a request completes successfully or unsuccessfully. In this case, upon a successful request, the aforementioned LiveData is updated with the response—a list of SearchResultItems. And thanks to the attached GsonConverterFactory, Retrofit automatically does the work of parsing the response body from JSON.

Like our first data source, FeedParser, FeedSearcher is also self-contained so that we can completely replace it with something else without affecting anything else in the app, although Feedly’s search engine works perfectly for my purposes. The most important thing it gives us is a collection of usable RSS feed URLS—any of these URLS can then be passed on to FeedParser to attempt to retrieve an actual RSS feed.

Building an RSS reader for Android from nothing: RSS feed retrieval and parsing

I’ve been meaning to write a series of posts about how I built NiceFeed—not a tutorial nor a step-by-step kind of thing, but more like a walkthrough of all the moving parts, how they came to be, and how they all fit together. I present it neither as an example of a great app nor of great programming—and naturally there are dozens of things that can be improved—but as a whole, I’m quite fond of it and use it everyday to get my news on my Android phone. My hope is to clarify some of my thinking about building the app and solving the many problems I encountered, so that it might be applied to future problems and projects. And of course, I hope that anybody who happens to stumble upon these posts may find them somehow instructive if not merely amusing.

RSS has been around for a long time and there are already many other readers and aggregators out there, but I’ve found many of them, particularly the free ones, awkward or difficult to navigate and jam-packed with features I don’t really need. My goal was an attractive and intuitive app, fully functional and with not too many frills.

*

An RSS feed appears in the wild as XML-formatted plain text, which needs to be parsed and converted to a form usable by an RSS aggregator application. I found that there were several third-party libraries for parsing RSS that already existed, so I didn’t have to write my own parsing code. I chose the aptly named RSS Parser because it seemed to be regularly updated and well documented. In addition, the library not only parses raw XML data, but executes its own HTTP requests, making it very easy to get up and running.

To start, I had to decide on what specific data I wanted the app to obtain. RSS Parser, after retrieving and parsing an RSS feed from the web, returns a Channel (or, the feed itself) object, which contains a collection of Articles (the contents of that feed). Both Channel and Article objects contain properties (title, author, URL, image, etc.) we definitely want and some that we don’t need. So, I created my own data classes, Feed and Entry, to contain only those properties that I wanted. Between the two is a many-to-many relationship: a feed is associated with many entries, and a single entry can be associated with more than one feed—we’ll return to this particular detail another time.

Determining which properties to include and exclude involved a lot of hemming and hawing as I became more familiar with the RSS data I was getting. I also added a few of my own: in Feed, category, which defaults to “Uncategorized,” and unreadCount, which simply holds the number of entries associated with that feed that have are not marked as read; and in Entry, the properties isStarred and isRead, both pretty self-explanatory and necessary if we want to be able to keep track of whether entries have been read and/or starred (or marked as favorite).

@Entity
data class Feed(
    @PrimaryKey val url: String, // Doubles as Feed ID
    var title: String,
    val website: String,
    val description: String? = null,
    val imageUrl: String? = null,
    var category: String = "Uncategorized",
    var unreadCount: Int
): Serializable

@Entity
data class Entry(
    @PrimaryKey val url: String, // Doubles as Entry ID
    val title: String,
    val website: String,
    val author: String?,
    val date: Date?,
    val content: String?,
    val image: String?,
    var isStarred: Boolean = false,
    var isRead: Boolean = false
) : Serializable {

    ...
}

Now for the real action. I created a class called FeedParser to contain all RSS Parser-related code. This class would contain all the needed methods for interacting with RSS Parser, and act as the authority across the entire app on retrieving and parsing RSS feeds from the internet. Naturally the class evolved over time as the app grew, but its role was always the same. Note: RSS Parser uses Kotlin Coroutines so the methods that use it have to be “suspend” functions.

/*  Responsible for retrieving and parsing RSS feeds */
class FeedParser(private val networkMonitor: NetworkMonitor) {

    private lateinit var rssParser: Parser
    private val _feedRequestLiveData = MutableLiveData<FeedWithEntries>()
    val feedRequestLiveData: LiveData<FeedWithEntries?>
        get() = _feedRequestLiveData

    suspend fun getFeedSynchronously(url: String): FeedWithEntries? {
        rssParser = Parser.Builder().build()
        return if (networkMonitor.isOnline) {
            try {
                val channel = rssParser.getChannel(url)
                ChannelMapper.makeFeedWithEntries(url, channel)
            } catch(e: Exception) {
                null
            }
        } else null
    }

    suspend fun requestFeed(url: String, backup: String? = null) {
        rssParser = Parser.Builder().build()
        if (networkMonitor.isOnline) {
            BackupUrlManager.setBase(backup)
            executeRequest(url)
        } else _feedRequestLiveData.postValue(null)
    }

    fun cancelRequest() {
        rssParser.cancel()
        BackupUrlManager.reset()
    }

    private suspend fun executeRequest(url: String) {
        // Automatically makes several requests with different possible URLs
        Log.d(TAG, "Requesting $url")

        try {
            val channel = rssParser.getChannel(url)
            val feedWithEntries = ChannelMapper.makeFeedWithEntries(url, channel)
            _feedRequestLiveData.postValue(feedWithEntries)
        } catch (e: Exception) {
            // If the initial request fails, try backup URL in different variations
            BackupUrlManager.getNextUrl()?.let { executeRequest(it) }
                ?: let {
                    _feedRequestLiveData.postValue(null)
                    Log.d(TAG, "Request failed")
                }
        }
    }

    ...

    companion object {

        private const val TAG = "FeedParser"
        private const val UNTITLED = "Untitled"
        const val FLAG_EXCERPT = "com.joshuacerdenia.android.nicefeed.excerpt "
    }
}

FeedParser takes a NetworkMonitor as an injected dependency. We’ll take a look at that at a later time, but it’s very simple: just an object that monitors the device’s internet connectivity. It contains one public property: isOnline, which at any given time is either true or false. All web requests are first checked against this property before executing.

There are three public methods, the first two of which are very similar, and begin with a new instance of RSS Parser:

First, getFeedSynchronously, which is really only for retrieving a feed as a background task. It takes a String called url (the address of an RSS feed), uses it to make a web request, and returns a FeedWithEntries: an additional data class that combines one Feed with a list of associated Entries. As it is meant to run in a background thread, the function waits for the web request to be completed before proceeding to the next step—hence, it is synchronous.

data class FeedWithEntries(
    @Embedded val feed: Feed,
    @Relation(
        parentColumn = "url",
        entityColumn = "url",
        associateBy = Junction(
            value = FeedEntryCrossRef::class,
            parentColumn = "feedUrl",
            entityColumn = "entryUrl"
        )
    )
    val entries: List<Entry>
)

Second, requestFeed, like the above, takes a String URL, and optionally a “backup” or second URL. Instead of returning anything, the main URL is passed first to a private method, executeRequest, which does the actual requesting via the current instance of RSS Parser. An object called BackupUrlManager notes the backup URL, if any, and hangs on to it until needed. We’ll take a closer look at it at a future time, but for now, all we need to know is that it generates different variations of the backup URL that can be used to retrieve a particular RSS feed.

Regarding the private method executeRequest: if there are any of these aforementioned variations of the backup URL, the method repeats itself via recursion until all variations are exhausted. Upon a successful request, the result is posted asynchronously to the class-level property feedRequestLiveData, which can then be read by whichever part of the app initiated the request.

The third public method is cancelRequest, which simply cancels the current instance of RSS Parser (and with it any pending request), and clears the BackupUrlManager.

You’ll notice also that in all of the above methods except cancelRequest, there is a reference to an object called ChannelMapper. As I said earlier, the RSS Parser object returns a Channel, which contains several properties as well as a collection of Articles. ChannelMapper is nested within FeedParser and contains methods for converting a Channel into a FeedWithEntries (again, a combination of one Feed and multiple Entry objects). This is just my way of organizing the code and keeping these methods in one place.

/*  Maps 'Channel' data into 'Feed' and 'Entry' objects */
private object ChannelMapper {

    private const val MAX_ENTRIES = 300 // Arbitrary
    private const val DATE_PATTERN = "EEE, d MMM yyyy HH:mm:ss Z"

    fun makeFeedWithEntries(url: String, channel: Channel): FeedWithEntries {
        val entries = mapEntries(channel, url)
        val feed = Feed(
            url = url, // The url that successfully completes the request is applied
            website = channel.link ?: url,
            title = channel.title ?: channel.link?.shortened() ?: url.shortened(),
            description = channel.description,
            imageUrl = channel.image?.url ?: channel.image?.link,
            unreadCount = entries.size
            )

        Log.d(TAG, "Retrieved ${entries.size} entries from $url")
        return FeedWithEntries(feed, entries)
    }

    private fun mapEntries(channel: Channel, url: String): List<Entry> {
        val entries = mutableListOf<Entry>()
        for (article in channel.articles) {
            if (entries.size < MAX_ENTRIES) {
                val entry = Entry(
                    url = article.link ?: article.guid ?: "",
                    website = channel.link ?: url,
                    title = article.title ?: UNTITLED,
                    author = article.author,
                    content = article.content ?: article.description.flagAsExcerpt(),
                    date = parseDate(article.pubDate),
                    image = article.image
                )
                entries.add(entry)
            } else break
        }
        return entries
    }

    private fun parseDate(stringDate: String?): Date? {
        return if (stringDate != null) {
            SimpleDateFormat(DATE_PATTERN, Locale.ENGLISH).parse(stringDate)
        } else null
    }

    private fun String?.flagAsExcerpt() = FLAG_EXCERPT + this
}

Here, the one public method, makeFeedWithEntries, accepts a String URL and Channel object and initiates the process of assigning all the data we need to properties that we specified earlier in the data classes Feed and Entry, and discarding the rest. The private method mapEntries, which makeFeedWithEntries calls within itself, does the same by looping through each Article contained in the Channel. In the end, we get a FeedWithEntries, ready to be stored or presented by the app.

Side note: at the bottom of ChannelMapper is a method flagAsExcerpt which extends a nullable String. I use it to flag any Entry whose content property is null and the description is not empty—in the wild, it means the entry is probably an excerpt. Many RSS feeds nowadays, especially from subscription sources, do not syndicate full versions of their content, only short excerpts. I have yet to do anything with flagged Entries, but might in the future: for example, the app could be made to open any Entry flagged as an excerpt automatically with the device’s default browser, instead of within the app.

And a final remark: I’ve written all my code with modularity and flexibility in mind, to the extent that I’m able. RSS Parser serves all my current needs but is not a perfect library—notably, it does not support Atom, and in the future I might want to use a different one. Since all the relevant code is contained entirely within the class FeedParser, we could easily create a new class with which to replace it as the final authority on all things related to retrieving and parsing RSS, without affecting much else in the app.

NiceFeed: Updating all subscriptions at once (And, picking up Python)

Per my earlier post, I had a nice break last week from working on NiceFeed while my internet was unusable. However, the plan to stay away from my computer turned out to be a miserable failure, and I decided instead to learn Python. I used Python Projects for Beginners: A Ten-Week Bootcamp Approach to Python Programming by Connor Milliken except instead instead of 10 weeks, I gave myself one. Of course, the real test is what to do with it now—I’d been planning to learn Django and pick up some backend skills, but the chapter on introductory data analytics really got my wheels spinning… We’ll see.

Anyway. A NiceFeed user emailed me about possibly adding an option to update all subscriptions at once. Up to this point, the closest thing I had was a background worker (via WorkManager) that cycles through all existing subscriptions but only updates one subscription every 15 minutes. If it finds any new content, it posts a notification displaying the most recent entry found. Needless to say, if a user has many, many subscriptions, it would take a long time for a full cycle to complete.

We want the app to be able to update all subscriptions in one move. This means quickly looping through every one of them, each time requesting the subscription URL, identifying any new content, and saving it to the database. To do this, I created a new CoroutineWorker class that does the following:


private val repo = NiceFeedRepository.get()
private val feedParser = FeedParser(repo.networkMonitor)

override suspend fun doWork(): Result {
    val feedUrls = repo.getFeedUrlsSynchronously()
    if (feedUrls.isEmpty()) return Result.success()

    for (url in feedUrls) {
        val currentEntryIds: List<String> = repo.getEntryIdsByFeedSynchronously(url)
        val feedWithEntries: FeedWithEntries? = feedParser.getFeedSynchronously(url)

        feedWithEntries?.entries?.let { entries ->
            val newEntries = entries.filterNot { currentEntryIds.contains(it.url) }
            val entryIds = entries.map { it.url }
            val oldEntryIds = currentEntryIds.filterNot { entryIds.contains(it) }
            repo.handleBackgroundUpdate(url, newEntries, oldEntryIds)
        }
    }

    return Result.success()
}

From there, it’s simply a matter of feeding it to WorkManager, which I’ve set to run the worker once a day. I’ve also added the option (via Snackbar message) to run the worker whenever new subscriptions are imported by OPML, since OPML only imports minimal feed data, without any entries. Now we are that much closer to a standard feature of most respectable RSS readers. I find that I enjoy “background” code like this much more than designing and creating the user interface… Hence, I’m still undecided as to whether I should also add a button to allow the option of running the worker whenever the user wants, rather than just on schedule. Thoughts for another day.

FizzBuzz, part 2: A fancier way

Piggybacking off my earlier post, because I have an obsessive personality I couldn’t help thinking of fancier, if more convoluted, ways to write FizzBuzz. I didn’t like the fact that I had to write the modulo operation and “Fizz” and “Buzz” more than once each. I came up with many variations of the code below (once again, in Kotlin):

val map = mapOf(3 to "Fizz", 5 to "Buzz")

fun main() {
    for (i in 1..100) {
        val fb = StringBuilder()
        map.forEach { if (i % it.key == 0) fb.append(it.value) }
        println(if (fb.isEmpty()) i else fb)
    }
}

Here the code uses a map (a set of key-value pairs) to establish the required relationships: 3 to Fizz, and 5 to Buzz. The outer loop, counting 1 through 100, is the same as before. We initialize an empty StringBuilder, and then start a second, inner loop that goes through each of the two aforementioned key-value pairs in the map and checks it against the current number i. If the operation returns true (that is, i is a multiple of the key of the current pair in the map), the value of the pair is added to the StringBuilder. So, if the operation returns true for the first key-value pair, we get “Fizz.” If only the second, “Buzz.” If both times, we get “FizzBuzz.” And each of these words appear in the code only once. Finally, we check if the string is empty in the end, and if so we just print the current number.

I could just as easily have done string concatenation by first assigning an empty string to a mutable variable (var fb = ""), but for reasons I don’t yet entirely understand, StringBuilder performs better, especially when loops are involved. In a program this small though, nobody will notice… I think?

Another beauty of this approach is that if we wanted to, we can change the rules very easily. We can add a third key-value pair to the map, maybe 7 to “Fuzz,” or use a different map entirely without rewriting anything else. Okay, I’m done with FizzBuzz now, moving on.

NiceFeed: An update, and taking a break

Due to current widespread disruption of internet services in the Philippines, I am taking a short break from NiceFeed through the end of the month, and plan to use the time to… stop staring at my computer screen all the time. But I’ll try to address somewhat urgent issues should they arise. 

Just today, a user wrote me about a crash in Android 6.0. I found the cause to be a few image resources that were mistakenly not accessible to API level 23 and below. Sometimes you get caught up in getting things right in all the obvious places, while big mistakes happen where you didn’t expect. After much fighting against my internet connection, I managed to roll out a quick update. It should be out on Google Play in a few days.

Also, the app has just been added to the IzzyOnDroid repo for the benefit of users outside the “walled garden”—as it has been described—of the Play Store (thanks for picking it up, IzzyOnDroid!). I appreciate the handful of users who have been trying out my app and taking the time to let me know of any issues or suggestions.

FizzBuzz: An intermezzo

Scaring myself with the revelation that many would-be developers ostensibly cannot write FizzBuzz, I became anxious to know if I would pass the test. The task is to write a program in as few characters as possible that will print the numbers 1 to 100 each on a new line, except multiples of 3 become “Fizz,” multiples of 5 are “Buzz,” and multiples of both 3 and 5 are “FizzBuzz.” As it turns out, the solution is nice and short in Kotlin:

fun main() {
    for (i in 1..100) { 
        println(when {
            i % 3 == 0 && i % 5 == 0 -> "FizzBuzz"
            i % 3 == 0 -> "Fizz"
            i % 5 == 0 -> "Buzz"
            else -> i
        })
    }
}

The overall logic is simple enough, just a plain old loop through 1 to 100 and conditionals to determine what to print. But I had to think for a second to remember the modulus operator… Here it means that when i is divided by either 3 or 5 and the remainder is 0, then it is a multiple of 3 or 5. I wonder if there’s an even more concise way to do this.

The when conditional makes things neat, instead of nested if/else if statements. i % 3 == 0 && i % 5 == 0 needs to come first, or else the program will print “Fizz” as long as i is a multiple of 3, regardless of whether it is also a multiple of 5. And so on. Another thing is that if i weren’t pressed for characters, I would maybe write "$i" in that else branch instead of just i to be consistent with the preceding branches, which all result in strings. But this appears to be a non-issue.