https://www.youtube.com/watch?v=2Oe9ZqXVGME&t=2427s How To Download and Process SEC XBRL Data Directly from EDGAR ^https://github.com/altova/sec-xbrl thanks Alex $ python loadSECfilings.py -y 2014 -m 1 –f 2014 –t 2017 http://rankandfiled.com/ pretty cool
Category: Computers & Technology
To run need to do two things (while in folder of python filename.py): 1) Run worker by executing the python program with the “worker” arguement: $ celery –A tasks worker –loglevel=info 2) Call the task aka run: $ python filename.py ####################### # grabhtml.py import requests from html import unescape class GrabHTML(object): def __init__(self): pass…
Python Script to insert CSV File into SQL Server Database import pandas as pd import csv import pyodbc import sys, os USERNAME = ‘sa’ PASSWORD = ‘password’ SERVER = ‘server’ DATABASE = ‘DATA’ DRIVERNAME = ‘ODBC Driver 13 for SQL Server’ cnxn = pyodbc.connect(‘Driver={‘+DRIVERNAME+’};Server=’+SERVER+’;Database=’+DATABASE+’;uid=’+USERNAME+’;pwd=’+PASSWORD) cur = cnxn.cursor() ##### using pandas grab first row for column…
Searching Website/Html/Document – currenly using Docfetcher, PowerGrep Flask/Django – looking into creating search page for html files Docker, MongoDB (3T MongoChef), Redis (Redis Desktop Manager) in Docker Storing the Data – HTML, CSV, JSON Pandas – dataframes, concat, merge OpenrefineGoogleRefine – Seemed to do trick so NLTK will come another day Natual Language Processing – Briefly…
Been been using ManicTime (www.manictime.com/) for >3 years to keep track of what I work on. Going to use this as source for weekly working on post.
ConEmu64.exe !@ !P -new_console:d -cur_console:znd:C:Usersryan C:WindowsSystem32bash.exe “ssh !U@!@ -p !# -t ‘cd !/ ; /bin/bash’” ^^ paste in WinSCP preferences edit 02-25-17: I just reloaded windows and realized that you need to install SSH.EXE in windows path (i.e. install git and make sure ssh.exe is in path… my bad) Now when connect, opens new console…
I constantly use docfetcher, indexing documents (pdfs, excel, word, html, etc DocFetcher website) and xplorer2 for viewing/storing results in scrap container (xplorer2 website). One thing that has been bugging me for a while is how when I copy a list of documents from DocFetcher I’ve had to copy and to excel before I could paste into…
Databases PostgreSQL 9 Redis MySQL MongoDB Cassandra (haven’t touched) Neo4j (Graphs) Data and Stream Processing Kafka Storm (haven’t touched) RabbitMQ Celery Puppet (haven’t touched) Text Search ElasticSearch Solr Tika Lucene Docfetcher (Tika & Lucene & Java) Web Scraping Scrapy Requests ?? Spark (haven’t touched) Hadoop (haven’t touched)