Automate Updates to an ArcGIS Online Hosted Feature Layer Using Python
Vložit
- čas přidán 18. 05. 2020
- Here is a long and ramble-y, unrefined video tutorial on how to automate updates for a hosted feature layer in ArcGIS Online using Python. I then show you how to schedule your Python script to run using Windows Task Scheduler. This uses the ArcGIS API for Python, which is free to download/use, and if you have ArcGIS Pro installed on your computer, you already have it set up, as it is included in the installation.
developers.arcgis.com/python/
The example downloads a zipped shapefile from on a public FTP site, but if you are using a file from your own computer or on a network drive, the basic workflow here still applies.
Additional documentation on this process can be found on the ESRI developers site:
developers.arcgis.com/python/...
Please note toward the end, when setting up the Task Scheduler, to choose 'Windows 10' at the bottom of the first tab for "Configure for:". In my video it was set to Windows Vista by default, which will prevent the task and script from running properly. - Jak na to + styl
Please note toward the end, when setting up the Task Scheduler, to choose 'Windows 10' at the bottom of the first tab for "Configure for:" right around the 55:40 mark. In my video it was set to Windows Vista and Server 2008 by default, which will prevent the task and script from running properly.
Thank you so much for the tutorial! It enabled me to implement an auto-update of a feature layer collection in our web map.
very clear video with a very practical case study. Thank you for this!
Great tutorial!!! I've been trying to find help on doing both of these things for a couple years now! hahah This was very helpful and clear! Thanks!
Thanks for this awesome tutorial that’s super easy to follow for people that don’t know how to code. So many tutorials skip over the basics and speak in advanced technical jargon. This was great!
This is great!!! Very easy to understand and use
Thank you so much Aaron for sharing this video
Super helpful video, thanks for sharing!
Thank you, Thank you!!!! This is extremely useful.
This is awesome - please make more videos like this, as you say - very little content in the way of tutorials for this kind of thing!
Thanks. I had plans for a couple more, but then ArcGIS Notebooks launched shortly after I made this video, which I then wanted to incorporate. However I was holding off for them to add a scheduling functionality. Hopefully it comes soon, though if I had known it would be this long I would have went ahead.
Great job!
It would be even more helpful to have an example of the process to add new rows to an existing hosted feature layer.
Man that was awesome, thanks a lot.
Thank you very easy to understand. Now to figure out how to do it with data I am creating from an API call. Thinking API -> csv -> overwrite or something like that
You could use pandas to read the CSV and put it in a spatial data frame, then move that to a feature set or feature collection and overwrite with this method or append using the edit features function. I think it’s under the geoaccessor portion of the ArcGIS api for python docs. Don’t ask for for a further explanation because I barely understand this stuff, I’ve just had some help creating a notebook at work that does that same type of workflow.
Thank you so much for great video. Can you please make a video how do you automate for CAD-files. And for files that have different projections system. Thanks in advance.!!!
great tutorial! just wondering, does the workflow still apply if i use google sheets?
You could adapt this to work with Google Sheets, just remove all the FTP stuff for downloading your file and instead look into something like 'gsheets' or the official help documents from Google on how to download a Sheet using python. Once you have the file downloaded, you can substitute it in as the file you use for the manager.overwrite step near the end of my example.
gsheets: pypi.org/project/gsheets/
Google Python/Sheets help document: developers.google.com/sheets/api/quickstart/python
Great tutorial.
If we have WFS service over hosted feature layer, will be also wfs service updated?
Thanks! Based on the ESRI documentation at the link below, it sounds like the WFS service should update automatically with the new data. See the 'Considerations' section, "Edits you make...". I haven't actually tried it myself, though. doc.arcgis.com/en/arcgis-online/manage-data/publish-wfs.htm
@@AaronKoelker Tnx. It works. Please, if you can, help me with some more info on how to prepare environment for work (what all needs to be installed, python, ArcGIS API for Python... is it all free) without installing arcgis pro (windows os). thank you
Hi Aaron -- thanks for the video, but that's not actually why I'm leaving this note. It's regarding your paperback rebinding video. No doubt you got lots of negative feedback from "real" bookbinders. I thought your approach was creative and good-looking, even if there were 3 or 4 or 5 small to medium shortcomings in the execution. If you're still interested in the possibilities, you might want to check out a video series called "Men in Sheds" (one of the subjects is a gifted art bookbinder).
Thanks for the recommendation, Robert. Fortunately I seem to have slipped under the radar of the negative feedback crowd, though I'm sure there are plenty of things I could improve upon. I only rebind books occasionally for myself and friends, and never do it to anything valuable.
Hello Aaron, Thank you, Great video. I am wondering whether it is possible to access all the hosted feature layers from the ArcGIS portal using Python API, I have accessed the feature layers but my goal is to access only the hosted feature layers in the portal.
You can use the same ArcGIS API for Python with Portal to access those hosted feature layers, but the update process in this video wouldn't work the same.
@@AaronKoelker Thank you, but I cannot get any documentation or a function to access the hosted feature layer, the only documentation is accessing using the url of the hosted feature layer or id of the layer. As I want to access all the hosted layers in the portal I cannot get anything regarding this.I can only access the feature layers in the portal. Kindly can you turn me into a right direction.
@@aravinthkumarb2557 So actually if you check out the second link in the video description, they are using Portal as an example. If your data is a hosted feature layer within the Portal then the same steps I used in the video will probably work, but I haven't actually tried it myself. If your Portal is a federated ArcGIS Server and the data is just a 'Feature Layer' referenced there, rather than a 'Hosted Feature Layer' actually stored within Portal, then it won't work.
Aaron great video! Can you show us how to download data from AGOL using scripts? Thanks man!
Thanks Jordan. Downloading AGOL data via python is super useful and pretty simple, if I have some time in the near future I might try to make a video for that, too. In the meantime, this example might point you in the right direction: support.esri.com/en/technical-article/000018909
A little late, but I have a video covering this now if you are still interested! czcams.com/video/yLaIR7lmyqw/video.html
Thanks, Aaron! Would you mind creating a GitHub repository to share the codes?
I haven't dived into the world of GitHub yet, but a lot of the code can be found on the developer site here: developers.arcgis.com/python/sample-notebooks/overwriting-feature-layers/
Hello Aaron, Nice work I wanna send email when feature is added in feature service? Kindly help me if you have any experience in this regards!
Check out the new webhook functionality you can do with hosted feature layers in ArcGIS Online www.esri.com/arcgis-blog/products/arcgis-online/sharing-collaboration/how-to-create-a-hosted-feature-service-webhook/
Hello, I get the following errors when I try to do this tutorial:
error 1: [Errno 11001] getaddrinfo failed
- un = gis.properties.user.username
error 2: gis: GIS
Method 'properties' has no 'user' memberpylint(no-member)
- dataItem = gis.content.get(itemid)
error 3: gis: GIS
Method 'content' has no 'get' memberpylint(no-member)
I am doing the tutorial with the latest version of Python, ArcGIS, and Visual Studio Code. Any help would be appreciated. Thanks.
Hm I'm not 100% sure what the issue is. I know the login process is slightly different if you are using any sort of enterprise logins for ArcGIS Online. Personally I've never gotten those kinds of accounts to work with the python api
@@AaronKoelker Thank you for your reply! I'm using an ArcGIS developer account. But, would you be able to share what version of PyCharm, Python, and ArcGIS Python API you were using for this video?
@@mariagonzalezbocanegra9000 I use a Developer account as well, so you should be good there. I believe it was Pycharm 2020.2 when I released the video, and I was using Python 3.6. The ArcGIS Python API version was 1.8.1. For your second error you can omit that line if you want, it's just for show to tell you who you logged in as but doesn't do anything practical.
Did you remember to do 'from arcgis.gis import GIS' toward the beginning of the script?
Great tutorial!!! I've been trying to find help on doing both of these things for a couple years now! hahah This was very helpful and clear! Thanks!