bridgy package
Reference documentation.
admin
Renders admin pages for ops and other management tasks.
Currently just /admin/responses
, which shows active responses with tasks that
haven’t completed yet.
- stats()[source]
Collect and report misc lifetime stats.
https://developers.google.com/appengine/docs/python/ndb/admin#Statistics_queries
Used to be on the front page, dropped them during the Flask port in August 2021.
blog_webmention
Converts webmentions to comments on Blogger, Tumblr, and WP.com.
- class BlogWebmentionView[source]
Bases:
Webmention
View for incoming webmentions against blog providers.
blogger
Blogger API 2.0 hosted blog implementation.
Blogger API docs: https://developers.google.com/blogger/docs/2.0/developers_guide_protocol
Python GData API docs: http://gdata-python-client.googlecode.com/hg/pydocs/gdata.blogger.data.html
To use, go to your Blogger blog’s dashboard, click Template, Edit HTML, then put this in the head section:
<link rel="webmention" href="https://brid.gy/webmention/blogger"></link>
https://developers.google.com/blogger/docs/2.0/developers_guide_protocol
https://developers.google.com/blogger/docs/2.0/developers_guide_protocol#CreatingComments
Test command line:
curl localhost:8080/webmention/blogger -d 'source=http://localhost/response.html&target=http://freedom-io-2.blogspot.com/2014/04/blog-post.html'
- class Blogger(*args, id=None, **kwargs)[source]
Bases:
Source
A Blogger blog.
The key name is the blog id.
- OAUTH_START
alias of
Start
- static new(auth_entity=None, blog_id=None, **kwargs)[source]
Creates and returns a Blogger for the logged in user.
- Parameters:
auth_entity (oauth_dropins.blogger.BloggerV2Auth)
blog_id (str) – which blog, optional; if not provided, uses the first available
- static urls_and_domains(auth_entity, blog_id=None)[source]
Returns an auth entity’s URL and domain.
- Parameters:
auth_entity (oauth_dropins.blogger.BloggerV2Auth)
blog_id – which blog. optional. if not provided, uses the first available.
- Returns:
([str url], [str domain])
browser
Browser extension views.
- merge_by_id(existing, updates)[source]
Merges two lists of AS1 objects by id.
Overwrites the objects in the existing list with objects in the updates list with the same id. Requires all objects to have ids.
- class BrowserSource(*args, id=None, **kwargs)[source]
Bases:
Source
A source whose data is provided by the browser extension.
Current subclasses are Instagram and Facebook.
- classmethod key_id_from_actor(actor)[source]
Returns the key id for this entity from a given AS1 actor.
To be implemented by subclasses.
- classmethod new(auth_entity=None, actor=None, **kwargs)[source]
Creates and returns an entity based on an AS1 actor.
- Parameters:
auth_entity – unused
actor (dict) – AS1 actor
- class BrowserView[source]
Bases:
View
Base class for requests from the browser extension.
- check_token(load_source=True)[source]
Loads the token and checks that it has at least one domain registered.
Expects token in the
token
query param.Raises (HTTPException): HTTP 403 if the token is missing or invalid
- class Status[source]
Bases:
BrowserView
Runs preflight checks for a source and returns status and config info.
- Response body is a JSON map with these fields:
status (str):
enabled
ordisabled
poll-seconds (int): current poll frequency for this source in seconds
- class Homepage[source]
Bases:
BrowserView
Parses a silo home page and returns the logged in user’s username.
Request body is https://www.instagram.com/ HTML for a logged in user.
- class Feed[source]
Bases:
BrowserView
Parses a silo feed page and returns the posts.
Request body is HTML from a silo profile with posts, eg https://www.instagram.com/name/ , for a logged in user.
Response body is the JSON list of translated ActivityStreams activities.
- class Profile[source]
Bases:
Feed
Parses a silo profile page and creates or updates its Bridgy user.
Request body is HTML from an IG profile, eg https://www.instagram.com/name/ , for a logged in user.
Response body is the JSON string URL-safe key of the Bridgy source entity.
- class Post[source]
Bases:
BrowserView
Parses a silo post’s HTML and creates or updates an Activity.
Request body is HTML from a silo post, eg https://www.instagram.com/p/ABC123/
Response body is the translated ActivityStreams activity JSON.
- class Extras[source]
Bases:
BrowserView
Merges extras (comments, reactions) from silo HTML into an existing Activity.
Requires the request parameter
id
with the silo post’s id (not shortcode!).Response body is the translated ActivityStreams JSON for the extras.
Subclasses must populate the
MERGE_METHOD
constant with the string name of the granary source class’s method that parses extras from silo HTML and merges them into an activity.
- class Comments[source]
Bases:
Extras
Parses comments from silo HTML and adds them to an existing Activity.
Requires the request parameter id with the silo post’s id (not shortcode!).
Response body is the translated ActivityStreams JSON for the comments.
- class Reactions[source]
Bases:
Extras
Parses reactions/likes from silo HTML and adds them to an existing Activity.
Requires the request parameter id with the silo post’s id (not shortcode!).
Response body is the translated ActivityStreams JSON for the reactions.
- class Poll[source]
Bases:
BrowserView
Triggers a poll for a browser-based account.
- class TokenDomains[source]
Bases:
BrowserView
Returns the domains that a token is registered for.
cron
Cron jobs. Currently just minor cleanup tasks.
- class LastUpdatedPicture(**kwargs)[source]
Bases:
StringIdModel
Stores the last user in a given silo that we updated profile picture for.
Key id is the silo’s
SHORT_NAME
.
- class UpdateFlickrPictures[source]
Bases:
UpdatePictures
Finds
Flickr
sources with new profile pictures and updates them.
- class UpdateMastodonPictures[source]
Bases:
UpdatePictures
Finds
Mastodon
sources with new profile pictures and updates them.
- class UpdateRedditPictures[source]
Bases:
UpdatePictures
Finds
Reddit
sources with new profile pictures and updates them.
facebook
Facebook API code and datastore model classes.
- class Facebook(*args, id=None, **kwargs)[source]
Bases:
BrowserSource
A Facebook account.
The key name is the Facebook global user id.
- OAUTH_START
alias of
Start
- classmethod new(auth_entity=None, actor=None, **kwargs)[source]
Creates and returns an entity based on an AS1 actor.
- classmethod key_id_from_actor(actor)[source]
Returns the actor’s numeric_id field to use as this entity’s key id.
numeric_id is the Facebook global user id.
- silo_url()[source]
Returns the Facebook profile URL, e.g. https://facebook.com/foo.
Facebook profile URLS with app-scoped user ids (eg www.facebook.com/ID) no longer work as of April 2018, so if that’s all we have, return None instead. https://developers.facebook.com/blog/post/2018/04/19/facebook-login-changes-address-abuse/
flickr
Flickr source and data model storage class.
- class Flickr(*args, id=None, **kwargs)[source]
Bases:
Source
A Flickr account.
The key name is the
nsid
.- OAUTH_START
alias of
Start
- AUTH_MODEL
alias of
FlickrAuth
- static new(auth_entity=None, **kwargs)[source]
Creates and returns a
Flickr
for the logged in user.- Parameters:
auth_entity (oauth_dropins.flickr.FlickrAuth)
- silo_url()[source]
Returns the Flickr account URL, e.g. https://www.flickr.com/people/foo/.
- user_tag_id()[source]
Returns the tag URI for this source, e.g. ‘tag:flickr.com:123456’.
- class Start(to_path, scopes=None)[source]
Bases:
Start
,AuthHandler
Custom handler to start Flickr auth process.
- class AddFlickr(to_path, scopes=None)[source]
Bases:
Callback
,AuthHandler
Custom handler to add Flickr source when auth completes.
If this account was previously authorized with greater permissions, this will trigger another round of auth with elevated permissions.
github
GitHub API code and datastore model classes.
- class GitHub(*args, id=None, **kwargs)[source]
Bases:
Source
A GitHub user.
The key name is the GitHub username.
- OAUTH_START
alias of
Start
- AUTH_MODEL
alias of
GitHubAuth
- static new(auth_entity=None, **kwargs)[source]
Creates and returns a
GitHub
for the logged in user.- Parameters:
auth_entity (oauth_dropins.github.GitHubAuth)
kwargs – property values
- silo_url()[source]
Returns the GitHub account URL, e.g. https://github.com/foo.
- user_tag_id()[source]
Returns this user’s tag URI, eg ‘tag:github.com:2013,MDQ6VXNlcjc3OD=’.
handlers
Common views, e.g. post and comment permalinks.
Docs: https://brid.gy/about#source-urls
URL paths are:
/post/SITE/USER_ID/POST_ID
e.g. /post/flickr/212038/10100823411094363/comment/SITE/USER_ID/POST_ID/COMMENT_ID
e.g. /comment/twitter/snarfed_org/10100823411094363/999999/like/SITE/USER_ID/POST_ID/LIKED_BY_USER_ID
e.g. /like/twitter/snarfed_org/10100823411094363/999999/repost/SITE/USER_ID/POST_ID/REPOSTED_BY_USER_ID
e.g. /repost/twitter/snarfed_org/10100823411094363/999999/rsvp/SITE/USER_ID/EVENT_ID/RSVP_USER_ID
e.g. /rsvp/facebook/212038/12345/67890
- class Item[source]
Bases:
View
Fetches a post, repost, like, or comment and serves it as mf2 HTML or JSON.
- get_item(**kwargs)[source]
Fetches and returns an object from the given source.
To be implemented by subclasses.
- Parameters:
source –
models.Source
subclassid – str
- Returns:
ActivityStreams object dict
- get_post(id, **kwargs)[source]
Fetch a post.
- Parameters:
id – str, site-specific post id
is_event – bool
kwargs – passed through to
get_activities()
- Returns:
ActivityStreams object dict
- merge_urls(obj, property, urls, object_type='article')[source]
Updates an object’s ActivityStreams URL objects in place.
Adds all URLs in urls that don’t already exist in
obj[property]
.ActivityStreams schema details: http://activitystrea.ms/specs/json/1.0/#id-comparison
indieauth
IndieAuth handlers for authenticating and proving ownership of a domain.
instagram
Instagram browser extension source class and views.
- class Instagram(*args, id=None, **kwargs)[source]
Bases:
BrowserSource
An Instagram account.
The key name is the username. Instagram usernames may have ASCII letters (case insensitive), numbers, periods, and underscores: https://stackoverflow.com/questions/15470180
- OAUTH_START
alias of
Start
- classmethod key_id_from_actor(actor)[source]
Returns the actor’s username field to be used as this entity’s key id.
- silo_url()[source]
Returns the Instagram account URL, e.g. https://instagram.com/foo.
mastodon
Mastodon source and datastore model classes.
- class StartBase(to_path, scopes=None)[source]
Bases:
Start
Abstract base OAuth starter class with our redirect URLs.
- class Mastodon(*args, id=None, **kwargs)[source]
Bases:
Source
A Mastodon account.
The key name is the fully qualified address, eg
@snarfed@mastodon.technology
.- AUTH_MODEL
alias of
MastodonAuth
- property URL_CANONICALIZER
Generated dynamically to use the instance’s domain.
- static new(auth_entity=None, **kwargs)[source]
Creates and returns a
Mastodon
entity.- Parameters:
auth_entity (oauth_mastodon.MastodonAuth)
kwargs – property values
- silo_url()[source]
Returns the Mastodon profile URL, e.g. https://foo.com/@bar.
medium
Medium hosted blog implementation.
Medium’s API is unsupported and degrading. We should probably sunset this.
Only supports outbound webmentions right now, not inbound, since Medium’s API doesn’t support creating responses or recommendations yet.
API docs:
- class Medium(*args, id=None, **kwargs)[source]
Bases:
Source
A Medium publication or user blog.
The key name is the username (with
@
prefix) or publication name.- OAUTH_START
alias of
Start
models
Datastore model classes.
- exception DisableSource[source]
Bases:
Exception
Raised when a user has deauthorized our app inside a given platform.
- class SourceMeta(name, bases, class_dict)[source]
Bases:
MetaModel
Source
metaclass. Registers all subclasses in thesources
global.
- class Source(*args, id=None, **kwargs)[source]
Bases:
StringIdModel
A silo account, e.g. a Facebook or Google+ account.
Each concrete silo class should subclass this class.
- classmethod new(**kwargs)[source]
Factory method. Creates and returns a new instance for the current user.
To be implemented by subclasses.
- classmethod lookup(id)[source]
Returns the entity with the given id.
By default, interprets id as just the key id. Subclasses may extend this to support usernames, etc.
Ideally, if
USERNAME_KEY_ID
, normalize to lower case before looking up. We’d need to backfill all existing entities with upper case key ids, though, which we’re not planning to do. https://github.com/snarfed/bridgy/issues/884
- silo_url(handler)[source]
Returns the silo account URL, e.g. https://twitter.com/foo.
- post_id(url)[source]
Resolve the ID of a post from a URL. By default calls out to Granary’s classmethod but can be overridden if a URL needs user-specific treatment.
- classmethod put_updates(source)[source]
Writes
source.updates
to the datastore transactionally.- Returns:
source (Source)
- Returns:
source
, updated
- poll_period()[source]
Returns the poll frequency for this source, as a
datetime.timedelta
.Defaults to ~30m, depending on silo. If we’ve never sent a webmention for this source, or the last one we sent was over a month ago, we drop them down to ~1d after a week long grace period.
- classmethod bridgy_webmention_endpoint(domain='brid.gy')[source]
Returns the Bridgy webmention endpoint for this source type.
- has_bridgy_webmention_endpoint()[source]
Returns True if this source uses Bridgy’s webmention endpoint.
- get_author_urls()[source]
Determine the author urls for a particular source.
In debug mode, replace test domains with localhost.
- get_activities_response(**kwargs)[source]
Returns recent posts and embedded comments for this source.
May be overridden by subclasses.
- get_comment(comment_id, **kwargs)[source]
Returns a comment from this source.
Passes through to granary by default. May be overridden by subclasses.
- Parameters:
comment_id (str) – site-specific comment id
kwargs – passed to
granary.source.Source.get_comment()
- Returns:
decoded ActivityStreams comment object, or None
- Return type:
- get_like(activity_user_id, activity_id, like_user_id, **kwargs)[source]
Returns an ActivityStreams
like
activity object.Passes through to granary by default. May be overridden by subclasses.
- Parameters:
activity_user_id (str) – id of the user who posted the original activity
activity_id (str) – activity id
like_user_id (str) – id of the user who liked the activity
kwargs – passed to
granary.source.Source.get_comment()
- create_comment(post_url, author_name, author_url, content)[source]
Creates a new comment in the source silo.
Must be implemented by subclasses.
- feed_url()[source]
Returns the RSS or Atom (or similar) feed URL for this source.
Must be implemented by subclasses. Currently only implemented by
blogger
,medium
,tumblr
, andwordpress_rest
.- Returns:
URL
- Return type:
- edit_template_url()[source]
Returns the URL for editing this blog’s template HTML.
Must be implemented by subclasses. Currently only implemented by
blogger
,medium
,tumblr
, andwordpress_rest
.- Returns:
URL
- Return type:
- format_for_source_url(id)[source]
Returns the given id formatted for a URL if necessary. Some silos use keys containing slashes. By default this is a no-op - can be overridden by subclasses.
- Parameters:
id – The id to format
- Returns:
string formatted id
- classmethod button_html(feature, **kwargs)[source]
Returns an HTML string with a login form and button for this site.
Mostly just passes through to
oauth_dropins.handlers.Start.button_html()
.- Returns:
HTML
- Return type:
- classmethod create_new(user_url=None, **kwargs)[source]
Creates and saves a new
Source
and adds a poll task for it.
- verified()[source]
Returns True if this source is ready to be used, False otherwise.
See
verify()
for details. May be overridden by subclasses, e.g.tumblr.Tumblr
.
- verify(force=False)[source]
Checks that this source is ready to be used.
For blog and listen sources, this fetches their front page HTML and discovers their webmention endpoint. For publish sources, this checks that they have a domain.
May be overridden by subclasses, e.g.
tumblr.Tumblr
.- Parameters:
force (bool) – if True, fully verifies (e.g. re-fetches the blog’s HTML and performs webmention discovery) even we already think this source is verified.
- urls_and_domains(auth_entity, user_url, actor=None, resolve_source_domain=True)[source]
Returns this user’s valid (not webmention-blocklisted) URLs and domains.
Converts the auth entity’s
user_json
to an ActivityStreams actor and uses itsurls
andurl
fields. May be overridden by subclasses.- Parameters:
- Return type:
([str url, …], [str domain, …]) tuple
- static resolve_profile_url(url, resolve=True)[source]
Resolves a profile URL to be added to a source.
- canonicalize_url(url, activity=None, **kwargs)[source]
Canonicalizes a post or object URL.
Wraps
oauth_dropins.webutil.util.UrlCanonicalizer
.
- infer_profile_url(url)[source]
Given a silo profile, tries to find the matching Bridgy user URL.
Queries Bridgy’s registered accounts for users with a particular domain in their silo profile.
- preprocess_for_publish(obj)[source]
Preprocess an object before trying to publish it.
By default this tries to massage person tags so that the tag’s
url
points to the person’s profile on this service (as opposed to a person’s homepage).The object is modified in place.
- Parameters:
obj (dict) – ActivityStreams activity or object
- on_new_syndicated_post(syndpost)[source]
Called when a new
SyndicatedPost
is stored for this source.- Parameters:
syndpost (SyndicatedPost)
- is_private()[source]
Returns True if this source is private aka protected.
…ie their posts are not public.
- is_beta_user()[source]
Returns True if this is a “beta” user opted into new features.
Beta users come from
beta_users.txt
.
- class Webmentions(**kwargs)[source]
Bases:
StringIdModel
A bundle of links to send webmentions for.
- class Response(**kwargs)[source]
Bases:
Webmentions
A comment, like, or repost to be propagated.
The key name is the comment object id as a tag URI.
- class Activity(**kwargs)[source]
Bases:
StringIdModel
An activity with responses to be propagated.
The key name is the activity id as a tag URI.
Currently only used for posts sent to us by the browser extension.
- class BlogPost(**kwargs)[source]
Bases:
Webmentions
A blog post to be processed for links to send webmentions to.
The key name is the URL.
- class PublishedPage(**kwargs)[source]
Bases:
StringIdModel
Minimal root entity for
Publish
children with the same source URL.Key id is the string source URL.
- class Publish(**kwargs)[source]
Bases:
Model
A comment, like, repost, or RSVP published into a silo.
Child of a
PublishedPage
entity.
- class BlogWebmention(**kwargs)[source]
Bases:
Publish
,StringIdModel
Datastore entity for webmentions for hosted blog providers.
Key id is the source URL and target URL concated with a space, ie
SOURCE TARGET
. The source URL is always the URL given in the webmention HTTP request. If the source page has au-url
, that’s stored in theu_url
property. The target URL is always the final URL, after any redirects.Reuses
Publish
’s fields, but otherwise unrelated.
- class SyndicatedPost(**kwargs)[source]
Bases:
Model
Represents a syndicated post and its discovered original (or not if we found no original post). We discover the relationship by following rel=syndication links on the author’s h-feed.
When a
SyndicatedPost
entity is about to be stored,source.Source.on_new_syndicated_post()
is called before it’s stored.- classmethod insert_original_blank(source, original)[source]
Insert a new original -> None relationship. Does a check-and-set to make sure no previous relationship exists for this original. If there is, nothing will be added.
- classmethod insert_syndication_blank(source, syndication)[source]
Insert a new syndication -> None relationship. Does a check-and-set to make sure no previous relationship exists for this syndication. If there is, nothing will be added.
- classmethod insert(source, syndication, original)[source]
Insert a new (non-blank) syndication -> original relationship.
This method does a check-and-set within transaction to avoid including duplicate relationships.
If blank entries exists for the syndication or original URL (i.e. syndication -> None or original -> None), they will first be removed. If non-blank relationships exist, they will be retained.
- Parameters:
- Returns:
newly created or preexisting entity
- Return type:
- class Domain(**kwargs)[source]
Bases:
StringIdModel
A domain owned by a user.
Ownership is proven via IndieAuth. Supports secret tokens associated with each domain. Clients can include a token with requests that operate on a given domain, eg sending posts and responses from the browser extension.
Key id is the string domain, eg
example.com
.
original_post_discovery
Augments the standard original_post_discovery algorithm with a reverse lookup that supports posts without a backlink or citation.
Performs a reverse-lookup that scans the activity’s author’s h-feed
for posts with rel=syndication links. As we find syndicated copies,
save the relationship. If we find the original post for the activity
in question, return the original’s URL.
See http://indiewebcamp.com/posse-post-discovery for more detail.
This feature adds costs in terms of HTTP requests and database lookups in the following primary cases:
Author’s domain is known to be invalid or blocklisted, there will be 0 requests and 0 DB lookups.
For a syndicated post has been seen previously (regardless of whether discovery was successful), there will be 0 requests and 1 DB lookup.
- The first time a syndicated post has been seen:
1 to 2 HTTP requests to get and parse the
h-feed
plus 1 additional request for each post permalink that has not been seen before.1 DB query for the initial check plus 1 additional DB query for each post permalink.
- discover(source, activity, fetch_hfeed=True, include_redirect_sources=True, already_fetched_hfeeds=None)[source]
Augments the standard original post discovery algorithm with a reverse lookup that supports posts without a backlink or citation.
If
fetch_hfeed
is False, then we will check the db for previously foundmodels.SyndicatedPost
s but will not do posse-post-discovery to find new ones.- Parameters:
source (Source) – subclass. Changes to property values (e.g. domains`,
domain_urls
,last_syndication_url
) are stored insource.updates
; they should be updated transactionally later.activity (dict)
fetch_hfeed (bool)
include_redirect_sources (bool) – whether to include URLs that redirect as well as their final destination URLs
already_fetched_hfeeds (set of str) – URLs that we have already fetched and run posse-post-discovery on, so we can avoid running it multiple times
- Returns:
(original post URLs, mention URLs)
- Return type:
- refetch(source)[source]
Refetch the author’s URLs and look for new or updated syndication links that might not have been there the first time we looked.
- Parameters:
source (Source) – Changes to property values (e.g.
domains
,domain_urls
,last_syndication_url
) are stored in source.updates; they should be updated transactionally later.- Returns:
mapping syndicated_url to a list of new
models.SyndicatedPost
s- Return type:
- targets_for_response(resp, originals, mentions)[source]
Returns the URLs that we should send webmentions to for a given response.
…specifically, all responses except posts get sent to original post URLs, but only posts and comments get sent to mentioned URLs.
- process_entry(source, permalink, feed_entry, refetch, preexisting, store_blanks=True)[source]
Fetch and process an h-entry and save a new
models.SyndicatedPost
.- Parameters:
source (Source)
permalink (str) – url of the unprocessed post
feed_entry (dict) – the
h-feed
version of theh-entry
, often contains a partial version of theh-entry
at the permalinkrefetch (bool) – whether to refetch and process entries we’ve seen before
preexisting (list) – of previously discovered
models.SyndicatedPost
s for this permalinkstore_blanks (bool) – whether we should store blank
models.SyndicatedPost
s when we don’t find a relationship
- Returns:
maps syndicated url to a list of new
models.SyndicatedPost
s- Return type:
pages
Bridgy user-facing pages: front page, user pages, delete POSTs, etc.
- users()[source]
View for
/users
.Semi-optimized. Pages by source name. Queries each source type for results with name greater than the start_name query param, then merge sorts the results and truncates at
PAGE_SIZE
.The start_name param is expected to be capitalized because capital letters sort lexicographically before lower case letters. An alternative would be to store a lower cased version of the name in another property and query on that.
- process_webmention_links(e)[source]
Generates pretty HTML for the links in a
models.Webmentions
entity.
publish
Publishes webmentions into the silos.
Webmention spec: http://webmention.org/
Bridgy request and response details: https://brid.gy/about#response
- exception CollisionError[source]
Bases:
RuntimeError
Multiple publish requests for the same page at the same time.
- class PublishBase[source]
Bases:
Webmention
Base handler for both previews and publishes.
Subclasses must set the
PREVIEW
attribute to True or False. They may also override other methods.- authorize()[source]
Returns True if the current user is authorized for this request.
Otherwise, should call
error()
to provide an appropriate error message.
- attempt_single_item(item)[source]
Attempts to preview or publish a single mf2 item.
- Parameters:
item (dict) – mf2 item from mf2py
- Return type:
- preprocess(activity)[source]
Preprocesses an item before trying to publish it.
Specifically, expands inReplyTo/object URLs with rel=syndication URLs.
- Parameters:
activity (dict) – ActivityStreams activity or object being published
- expand_target_urls(activity)[source]
Expand the inReplyTo or object fields of an ActivityStreams object by fetching the original and looking for rel=syndication URLs.
This method modifies the dict in place.
- Parameters:
activity (dict) – ActivityStreams activity being published
- get_or_add_publish_entity(source_url)[source]
Creates and stores
models.Publish
entity.…and if necessary,
models.PublishedPage
entity.- Parameters:
source_url (str)
- class Preview[source]
Bases:
PublishBase
Renders a preview HTML snippet of how a webmention would be handled.
- class Send[source]
Bases:
PublishBase
Interactive publish handler. Redirected to after each silo’s OAuth dance.
- class Webmention[source]
Bases:
PublishBase
Accepts webmentions and translates them to publish requests.
reddit
Reddit source code and datastore model classes.
- class Reddit(*args, id=None, **kwargs)[source]
Bases:
Source
A Reddit account.
The key name is the username.
- OAUTH_START
alias of
Start
- static new(auth_entity=None, **kwargs)[source]
Creates and returns a
Reddit
entity.- Parameters:
auth_entity (oauth_dropins.reddit.RedditAuth)
kwargs – property values
- silo_url()[source]
Returns the Reddit account URL, e.g. https://reddit.com/user/foo.
- get_activities_response(*args, **kwargs)[source]
Set user_id manually.
…since Reddit sometimes (always?) 400s our calls to https://oauth.reddit.com/api/v1/me (via PRAW’s Reddit.user.me() ).
superfeedr
Superfeedr.
- subscribe(source)[source]
Subscribes to a source.
Also receives some past posts and adds propagate tasks for them.
http://documentation.superfeedr.com/subscribers.html#addingfeedswithpubsubhubbub
- Parameters:
source (Blogger Tumblr, or WordPress)
- handle_feed(feed, source)[source]
Handles a Superfeedr JSON feed.
Creates
models.BlogPost
entities and adds propagate-blogpost tasks for new items.
- class Notify[source]
Bases:
View
Handles a Superfeedr notification.
Abstract; subclasses must set the
SOURCE_CLS
attr.http://documentation.superfeedr.com/subscribers.html#pubsubhubbubnotifications
tasks
Task queue handlers.
- class Poll[source]
Bases:
View
Task handler that fetches and processes new responses from a single source.
Request parameters:
source_key
: string key of source entitylast_polled
: timestamp,YYYY-MM-DD-HH-MM-SS
Inserts a propagate task for each response that hasn’t been seen before.
Steps:
Fetch activities: posts by the user, links to the user’s domain(s).
Extract responses, store their activities.
Filter out responses we’ve already seen, using
models.Response
s in the datastore.Store new responses and enqueue propagate tasks.
Possibly refetch updated syndication urls.
1-4 are in
backfeed()
; 5 is inpoll()
.- poll(source)[source]
Actually runs the poll.
Stores property names and values to update in
source.updates
.
- class Discover[source]
Bases:
Poll
Task handler that fetches and processes new responses to a single post.
Request parameters:
source_key (string): key of source entity
post_id (string): silo post id(s)
Inserts a propagate task for each response that hasn’t been seen before.
- class SendWebmentions[source]
Bases:
View
Abstract base task handler that can send webmentions.
Attributes:
entity (models.Webmentions): subclass instance (set in
lease_entity()
)source (models.Source): entity (set in
send_webmentions()
)
- source_url(target_url)[source]
Return the source URL to use for a given target URL.
Subclasses must implement.
- Parameters:
target_url (str)
- Returns:
str
- send_webmentions()[source]
Tries to send each unsent webmention in self.entity.
Uses
source_url()
to determine the source parameter for each webmention.lease()
must be called before this!
- lease(key)[source]
Attempts to acquire and lease the
models.Webmentions
entity.Also loads and sets
g.source
, and returns False if the source doesn’t exist or is disabled.- Parameters:
key (ndb.Key)
- Returns:
True on success, False or None otherwise
- Return type:
- complete()[source]
Attempts to mark the
models.Webmentions
entity completed.Returns True on success, False otherwise.
- release(new_status)[source]
Attempts to unlease the
models.Webmentions
entity.- Parameters:
new_status (str)
- class PropagateResponse[source]
Bases:
SendWebmentions
Task handler that sends webmentions for a
models.Response
.Attributes:
activities: parsed
models.Response.activities_json
list
Request parameters:
response_key (str): key of
models.Response
entity
- class PropagateBlogPost[source]
Bases:
SendWebmentions
Task handler that sends webmentions for a
models.BlogPost
.Request parameters:
key (str): key of
models.BlogPost
entity
tumblr
Tumblr + Disqus blog webmention implementation.
To use, go to your Tumblr dashboard, click Customize, Edit HTML, then put this in the head section:
<link rel="webmention" href="https://brid.gy/webmention/tumblr">
Misc notes and background:
Guest post (w/arbitrary author, url):
http://spirytoos.blogspot.com/2013/12/not-so-easy-posting-as-guest-via-disqus.html
http://stackoverflow.com/questions/15416688/disqus-api-create-comment-as-guest
http://jonathonhill.net/2013-07-11/disqus-guest-posting-via-api/
Can send url and not look up disqus thread id!
Test command line:
curl localhost:8080/webmention/tumblr -d 'source=http://localhost/response.html&target=http://snarfed.tumblr.com/post/60428995188/glen-canyon-http-t-co-fzc4ehiydp?foo=bar#baz'
- class Tumblr(*args, id=None, **kwargs)[source]
Bases:
Source
A Tumblr blog.
The key name is the blog domain.
- OAUTH_START
alias of
Start
- static new(auth_entity=None, blog_name=None, **kwargs)[source]
Creates and returns a
Tumblr
for the logged in user.- Parameters:
auth_entity (oauth_dropins.tumblr.TumblrAuth)
blog_name (str) – which blog, optional, passed to
urls_and_domains()
- static urls_and_domains(auth_entity, blog_name=None)[source]
Returns this blog’s URL and domain.
- Parameters:
auth_entity (oauth_dropins.tumblr.TumblrAuth)
blog_name (str) – which blog, optional, matches the
name
field for one of the blogs inauth_entity.user_json['user']['blogs']
- Return type:
([str url], [str domain])
- verify()[source]
Checks that Disqus is installed as well as the webmention endpoint.
Stores the result in webmention_endpoint.
twitter
Twitter source code and datastore model classes.
The Twitter API is dead, and so is this code.
- class Twitter(*args, id=None, **kwargs)[source]
Bases:
Source
A Twitter account.
The key name is the username.
- OAUTH_START
alias of
Start
- AUTH_MODEL
alias of
TwitterAuth
- static new(auth_entity=None, **kwargs)[source]
Creates and returns a
Twitter
entity.- Parameters:
auth_entity (oauth_dropins.twitter.TwitterAuth)
kwargs – property values
- silo_url()[source]
Returns the Twitter account URL, e.g. https://twitter.com/foo.
- search_for_links()[source]
Searches for activities with links to any of this source’s web sites.
Twitter search supports OR: https://dev.twitter.com/rest/public/search
…but it only returns complete(ish) results if we strip scheme from URLs, ie search for example.com instead of http://example.com/, and that also returns false positivies, so we check that the returned tweets actually have matching links. https://github.com/snarfed/bridgy/issues/565
- Returns:
sequence of ActivityStreams activity dicts
- get_like(activity_user_id, activity_id, like_user_id, **kwargs)[source]
Returns an ActivityStreams ‘like’ activity object for a favorite.
We get Twitter favorites by scraping HTML, and we only get the first page, which only has 25. So, use a
models.Response
in the datastore first, if we have one, and only re-scrape HTML as a fallback.- Parameters:
activity_user_id (str) – id of the user who posted the original activity
activity_id (str) – activity id
like_user_id (str) – id of the user who liked the activity
kwargs – passed to
granary.source.Source.get_comment()
- class Start(to_path, scopes=None, access_type=None)[source]
Bases:
Start
,Auth
Custom OAuth start handler that uses
access_type=read
forstate=listen
.Tweepy converts access_type to
x_auth_access_type
for Twitter’s oauth/request_token endpoint. Details: https://dev.twitter.com/docs/api/1/post/oauth/request_token
util
Misc utility constants and classes.
- add_poll_task(source, now=False)[source]
Adds a poll task for the given source entity.
Pass
now=True
to insert apoll-now
task.
- add_propagate_blogpost_task(entity)[source]
Adds a propagate-blogpost task for the given response entity.
- add_discover_task(source, post_id, type=None)[source]
Adds a discover task for the given source and silo post id.
- add_task(queue, eta_seconds=None, **kwargs)[source]
Adds a Cloud Tasks task for the given entity.
- Parameters:
queue (str) – queue name
entity (Source or Webmentions)
eta_seconds (int) – optional
kwargs – added to task’s POST body (form-encoded)
- exception Redirect(new_url: str)[source]
Bases:
RequestRedirect
Adds login cookie support to
werkzeug.exceptions.RequestRedirect
.
- redirect(path, code=302, logins=None)[source]
Stops execution and redirects to the absolute URL for a given path.
Specifically, raises
werkzeug.routing.RequestRedirect
.
- webmention_endpoint_cache_key(url)[source]
Returns cache key for a cached webmention endpoint for a given URL.
Example:
https snarfed.org /
If the URL is the home page, ie path is
/
, the key includes a/
at the end, so that we cache webmention endpoints for home pages separate from other pages. https://github.com/snarfed/bridgy/issues/701
- report_error(msg, **kwargs)[source]
Reports an error to StackDriver Error Reporting.
https://cloud.google.com/error-reporting/docs/reference/libraries#client-libraries-install-python
- Parameters:
msg (str)
- requests_get(url, **kwargs)[source]
Wraps
requests.get()
with extra semantics and our user agent.If a server tells us a response will be too big (based on
Content-Length
), we hijack the response and return 599 and an error response body instead. We passstream=True
torequests.get()
so that it doesn’t fetch the response body until we accessrequests.Response.content
(orrequests.Response.text
).http://docs.python-requests.org/en/latest/user/advanced/#body-content-workflow
- fetch_mf2(url, **kwargs)[source]
Injects
requests_get()
intooauth_dropins.webutil.util.fetch_mf2()
.
- requests_post(url, **kwargs)[source]
Wraps
requests.post()
with our headers.
- follow_redirects(url)[source]
Wraps
oauth_dropins.webutil.util.follow_redirects()
with our headers.
- get_webmention_target(url, resolve=True, replace_test_domains=True)[source]
Resolves a URL and decides whether we should try to send it a webmention.
Note that this ignores failed HTTP requests, ie the boolean in the returned tuple will be True! TODO: check callers and reconsider this.
- Parameters:
- Returns:
target info. should send is True if we should send a webmention, False otherwise, eg if it’s a bad URL, not
text/html
or in the blocklist.- Return type:
(str url, str pretty domain, bool should send) tuple
- in_webmention_blocklist(domain)[source]
Returns True if the domain or its root domain is in
BLOCKLIST
.
- is_opt_out(actor)[source]
Whether this user has explicitly opted out of Bridgy.
Currently just looks for
#nobridge
or#nobot
in profile description/bio.https://github.com/snarfed/bridgy/issues/1598
Duplicates
Object.status
in Bridgy Fed!
- prune_activity(activity, source)[source]
Prunes an activity down to just id, url, content, to, and object, in place.
If the object field exists, it’s pruned down to the same fields. Any fields duplicated in both the activity and the object are removed from the object.
Note that this only prunes the to field if it says the activity is public, since
as1.is_public()
defaults to saying an activity is public if the to field is missing. If that ever changes, we’ll need to start preserving the to field here.
- replace_test_domains_with_localhost(url)[source]
Replace domains in
LOCALHOST_TEST_DOMAINS
with localhost for testing.
- load_source(error_fn=None)[source]
Loads a source from the
source_key
orkey
query parameter.Expects the query parameter value to be a URL-safe key. Returns HTTP 400 if neither parameter is provided or the source doesn’t exist.
- Parameters:
error_fn (callable) – to be called with errors. Takes one parameter, the string error message.
- Return type:
- maybe_add_or_delete_source(source_cls, auth_entity, state, **kwargs)[source]
Adds or deletes a source if
auth_entity
is not None.Used in each source’s oauth-dropins
Callback.finish()
andCallback.get()
methods, respectively.This method _always_ ends by raising a
Redirect
exception to return an HTTP redirect response! That means that any code after a call to this function will not run!- Parameters:
source_cls (granary.source.Source subclass) – eg
granary.instagram.Instagram
auth_entity (oauth_dropins.models.BaseAuth subclass instance) – auth entity
state (str) – OAuth callback
state
parameter. a JSON serialized dict withoperation
,feature
, and an optional callback URL. For deletes, it will also include the source keykwargs – passed through to the
source_cls
constructor
- Returns:
source entity if it was created or updated, otherwise None
- Raises:
Redirect – to redirect to user page or callback URL
- construct_state_param_for_add(state=None, **kwargs)[source]
Construct the state parameter if one isn’t explicitly passed in.
The following keys are common:
operation
:add
ordelete
feature
:listen
,publish
, orwebmention
callback
: an optional external callback URL that we will redirect to at the end of the authorization handshakesource
: the source entity key, only applicable to deletes
- get_logins()[source]
Extracts the current user page paths from the logins cookie.
The logins cookie is set in
redirect()
andRedirect
.
- preprocess_source(source)[source]
Prepares a source entity for rendering in the source.html template.
converts image URLs to https if we’re serving over SSL
sets
website_links
attr to list of pretty HTML links to domain_urls
- Parameters:
source (Source) – entity
- oauth_starter(oauth_start_view, **kwargs)[source]
Returns an oauth-dropins start view that injects the state param.
- Parameters:
oauth_start_view – (oauth_dropins.views.Start subclass instance): eg
oauth_dropins.twitter.Start
.kwargs – passed to
construct_state_param_for_add()
- unwrap_t_umblr_com(url)[source]
If url is a
t.umblr.com
short link, extract its destination URL.Otherwise, return
url
unchanged.Not in
tumblr.py
sincemodels
importssuperfeedr
, so it would be a circular import.Background: https://github.com/snarfed/bridgy/issues/609
webmention
Base handler class and common utilities for handling webmentions.
Used in publish.py and blog_webmention.py.
Webmention spec: http://webmention.org/
- webmention_get_or_head(silo)[source]
Serves webmention discovery for HEADs to webmention endpoints.
- class Webmention[source]
Bases:
View
Webmention base view.
Attributes: * source (models.Source): for this webmention * entity (models.Publish or models.Webmention) entity for this webmention
- fetch_mf2(url, id=None, require_mf2=True, raise_errors=False)[source]
Fetches a URL and extracts its mf2 data.
Side effects: sets
entity.html
on success, callserror
on errors.- Parameters:
url – str
id – str, optional id of specific element to extract and parse. defaults to the whole page.
require_mf2 – boolean, whether to return error if no mf2 are found
raise_errors – boolean, whether to let error exceptions propagate up or handle them
- Return type:
- error(error, html=None, status=400, data=None, log_exception=False, report=False, extra_json=None, http_response=True)[source]
Handle an error. May be overridden by subclasses.
- Parameters:
error (str) – human-readable error message
html (str) – HTML human-readable error message
status (int) – HTTP response status code
data (dict) – mf2 data parsed from source page
log_exception (bool) – whether to include a stack trace in the log msg
report (bool) – whether to report to StackDriver Error Reporting
extra_json (dict) – to be merged into the JSON response body
http_response (bool) – whether to returning an error HTTP response
wordpress_rest
WordPress REST API (including WordPress.com) hosted blog implementation.
To use, go to your WordPress.com blog’s admin console, then go to Appearance, Widgets, add a Text widget, and put this in its text section:
<a href="https://brid.gy/webmention/wordpress" rel="webmention"></a>
Not this, it breaks:
<link rel="webmention" href="https://brid.gy/webmention/wordpress">
https://developer.wordpress.com/docs/api/
Create returns id, can lookup by id.
Test command line:
curl localhost:8080/webmention/wordpress -d 'source=http://localhost/response.html&target=http://ryandc.wordpress.com/2013/03/24/mac-os-x/'
Making an API call with an access token from the command line:
curl -H 'Authorization: Bearer [TOKEN]' URL...
- class WordPress(*args, id=None, **kwargs)[source]
Bases:
Source
A WordPress blog.
The key name is the blog hostname.
- OAUTH_START
alias of
Start
- static new(auth_entity=None, **kwargs)[source]
Creates and returns a WordPress for the logged in user.
- Parameters:
auth_entity (oauth_dropins.wordpress_rest.WordPressAuth)
- urls_and_domains(auth_entity)[source]
Returns this blog’s URL and domain.
- Parameters:
auth_entity – unused
- Return type:
([str url], [str domain]) tuple
- create_comment(post_url, author_name, author_url, content)[source]
Creates a new comment in the source silo.
If the last part of the post URL is numeric, e.g.
http://site/post/123999
, it’s used as the post id. Otherwise, we extract the last part of the path as the slug, e.g.http://site/post/the-slug
, and look up the post id via the API.