#datamigration

November 14, 2022 9:19am

How to migrate data from WordPress to Django?

Hi all, in my job training i'm working on creating a new version of the smart map

The current website is using WordPress for the backend and the version i'm working on uses Django. I'd like to migrate data from the WordPress database into Django

From what i've been able to gather there are these alternatives:
* Using Django's inspectdb (as described here)
* Exporting the WP data to XML and then parsing this data with a custom script (maybe something like this)

What would you recommend? Are there other options?

#WordPress #Django #DataMigration #programming #databases

Doc Morbius

April 1, 2022 5:41pm

Socialhome.Network as a searchable federation archive

The Socialhome node https://socialhome.network/ is both widely federated AND searchable by non-members. It displays both posts AND profile pages even to non-members, which is NOT the case for Diaspora* pods.

If you're looking for specific content from a now-defunct pod, Socialhome may be able to turn that up for you.

You can search for a specific profile by the its handle, e.g., dredmorbius@diaspora.glasswings.com or dredmorbius@joindiaspora.com, for my present and now-defunct Joindiaspora profiles, using the site's Search feature.

My (now-offline at origin) Joindiaspora profile appears as:

https://socialhome.network/p/702b2f0c-0f8f-4e17-b308-24a43f9ebed9/

Note that the profile GUID is NOT the same as it would be for my Joindiaspora profle itself (d8210c0de509264f`). The Diaspora* GUID can be used to construct a URL visible from the Diaspora* Pod you have an account on and are logged in to, but not as a globally-viewable third-party-accessible URL. Socialhome solves this problem.

That is, if you are on Glasswings, this URL links to my Joindiaspora profile:

https://diaspora.glasswings.com/people/d8210c0de509264f

But if you're not, it won't. Glasswings users might try a different pod such as Diasp.org:

https://diasp.org/people/d8210c0de509264f

Instead, third-party visitors are presented with a log-in / registration page. Socialhome solves this problem specifically.

There may be additional features / API apparent at the Socialhome GitLab repo: https://git.feneas.org/socialhome/socialhome

Noting that that is a FENEAS URL, also likely to go offline in the near future, GitHub:

https://github.com/jaywink/socialhome

Limitations

The downsides to Socialhome seem to be that:

References to content and profiles does not follow Diaspora*-assigned GUIDs. That is, there's no automated way to refernece a specific post or profile.
I'm not seeing an obvious way of exposing a JSON abstract of posts or profiles --- the Diaspora* trick of appending .json to the end of a URL does not work, and I'm not seeing a JSON abstract in the raw HTML. There may still be an API.

Photos are displayed but appear to be served from the original host, not separately federated. Once the origin host goes offline, photo links / embeds will be broken.

It's not clear how widely or deeply content is federated, though some should be better than none.

This option was brought to my attention by @Isaac Kuo in comments here.

#Diaspora #DataArchival #Federation #SocialHome #Pluspora #Joindiaspora #DataMigration #Archives #Plexodus #GooglePlus #GPlus #FENEAS

Doc Morbius

March 30, 2022 6:40pm

#Pluspora Final Days --- Shutdown likely on 1 April 2022

For those who are still on the Pluspora Pod, final shutdown will be imminent in the next days. The planned shutdown date is 1 April 2022 so far as I'm aware.

The announcement and updates are here:

FOLLOW UP ON PLUSPORA POD CLOSURE

https://sites.google.com/talesfrombabylon.com/pluspora/home

Announcement on Diaspora:

"It is with a heavy heart that we come to you with this post."

Federated copies:
- https://diasp.org/posts/f09b32d07568013a3ddf005056264835
- https://diasp.eu/posts/f09b32d07568013a3ddf005056264835
- https://diaspora.glasswings.com/posts/f09b32d07568013a3ddf005056264835

Archives: post:

https://archive.is/cuztp https://archive.is/rHhLP https://archive.ph/h3aw4

Things you should do NOW

Create a new profile elsewhere if you have not already. Diasp.org, Glasswings, and several other pods have welcomed well-behaved members. I can offer invites to Glasswings on request.
Tell others where you've gone, and where you've come from. That is, leave a FORWARDING address on your old Pluspora profile, and a PREVIOUS address on your new profile elsewhere. Posts tagged with #SIGNALFLARE will be easier for others to find as well.
If you have any thought that you might want to retain your old posts / content, archive that NOW. Note that given the passing of David Thiery, support for failed exports is unlikely, but you can at least try. Content cannot presently be imported to a Diaspora pod, but may be in the future. I've created numerous tools for working with that data described in posts below.
Federation will preserve at least some of your content on Diaspora*. This means that having followers on other Pods is highly useful. I've explored the possibilities and limits of this further elsewhere.
I've created tools and methods for archiving content, including automated submission of posts to the Internet Archive, and manual submission to Archive Today. Both rely on having a downloaded archive of your older content.
Follower / Block lists are not importable. You'll have to re-establish contact with people generally manually. Given that many Pluspora contacts were on Pluspora itself ... this may be complicated. The regular #Checkin posts are an excellent way to get back in touch with the community.
If you want your Profile Bio Page to be world-readable, you need to set that specifically at /profile/edit. Toggle "Visibility of your extended profile" to "Public".
If nothing else, you can save your Profile page at https://pluspora.com/u/<username> to the Internet Archive and Archive Today manually by visiting https://web.archive/org/save/ and https://archive.today. Do this after leaving a forwarding address to make your new profile easier to find.
Please do NOT delete your profile or posts if you wish for federated copies to remain available elsewhere. Unfortunately this is something of an unclean break. Deleting your profile will delete all your posts / comments to other pods.

Resources

Continuation / informational sites

There are a Pluspora Wiki and a Twitter group:

Data Migration Posts

The following posts offer tools and lessons learned in my own recent Pod migration.

Diaspora* Data Migration and Archival Lessons Learned

This is a summary of my discoveries and learning over the past two months or so concerning Diaspora* data archives and references as well as JSON and tools for manipulating it, specifically jq. ...

https://diaspora.glasswings.com/posts/fbf5b8f077f9013a6e0d448a5b29e257

Diaspora Migration Tips and Questions Thread

I’m going to be posting suggestions and questions regarding migrating between and/or preserving Diaspora* profile data....

https://diaspora.glasswings.com/posts/e0b4d980545c013a0103448a5b29e257

Diaspora* Migration & Data Archival: Finding your First Remote-Pod Followers

Federated content is likely to be found on remote pods from which profiles first followed you. You can find your earlierst remote-pod followers by going through your Notifications. Just note the pod.

This content may not be universally available to others, but offers some alternatives to archival in a pinch.

https://diaspora.glasswings.com/posts/feaae8207c8d013a5b1e448a5b29e257

As a reminder: I was not a Pluspora member, though I'd revived my Diaspora* profile with the shutdown of Google+ in 2019, so many, many, many centuries ago... I did move recently from the now-defunct #Joindiaspora pod to my new home on Glasswings.

#DataMigration #DiasporaMigration #Pluspora #GooglePlus #Diaspora #DiasporaMigration #Migration #Diaspora #Help #Tips #DataArchves #Archives #Plexodus

A Pluspora Community!☕https://t.co/XYPIvdu7Ho
— Nathan Weaver (@babylontales) February 19, 2022

Doc Morbius

March 5, 2022 7:18am

Internet Archive's Wayback Machine APIs

Say, for the sake of argument, that you've^†^ spent a few weeks trying to ensure that a set of URLs were archived at the Internet Archive's Wayback Machine, and you're aware that not all of those URLs were in fact archived, but you^†^ aren't sure just which ones were or were not.

The question might occur to you^†^, "Is there some way of testing whether or not a particular URL has or has not been successfully archived? Preferably in an automated manner?"

And the answer to that question would be YES!!! Yes there is!!!

What you^†^ are looking for are the Internet Archive Wayback Machine APIs, and specifically:

Wayback Availability JSON API

This simple API for Wayback is a test to see if a given url is archived and currenlty accessible in the Wayback Machine. This API is useful for providing a 404 or other error handler which checks Wayback to see if it has an archived copy ready to display.

Quoting from the Archive:

The API can be used as follows:

http://archive.org/wayback/available?url=example.com

which might return:

{
    "archived_snapshots": {
        "closest": {
            "available": true,
            "url": "http://web.archive.org/web/20130919044612/http://example.com/",
            "timestamp": "20130919044612",
            "status": "200"
        }
    }
}

if the url is available. When available, the url is the link to the archived snapshot in the Wayback Machine At this time, archived_snapshots just returns a single closest snapshot, but additional snapshots may be added in the future.

If the url is not available (not archived or currently not accessible), the response will be:

{"archived_snapshots":{}}

https://archive.org/help/wayback_api.php

It's also possible to query for a specific timestamp, though not AFAICT for saves within a date range.

You^†^ are now running that check on a set of 1300 or so URLs you'd^†^ hoped to have saved in the past two months or so.

See Also: Data Migration Tips and Questions.

Notes:

†: And my "you" I of course me "me".

#DataMigration #InternetArchive #APIs #Joindiaspora #JoindiasporaCom #Pluspora #Archival

Doc Morbius (moved to Glasswings)

October 23, 2018 4:45pm

A reminder: REQUEST JSON FORMAT for Google Data Takeout

Google HTML is absolutely worthless.

The instructions don't make this clear, but you're going to see this advice repeated repeatedly and repetitiously. With great frequency. Repeatedly.

Distracted JSON