Migrate to PostgreSQL #39
No reviewers
Labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Miroslavsckaya/tg_rss_bot#39
Loading…
Reference in a new issue
No description provided.
Delete branch "migrate_to_postgresql"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Migrate to PostgreSQL from sqlite
Closes #15
Closes #32
@ -11,2 +11,2 @@
token = os.getenv('TELEGRAM_TOKEN')
db_path = os.getenv('DATABASE_PATH', './bot.db')
token = os.getenv('RSSBOT_TG_TOKEN')
db_dsn = os.getenv('RSSBOT_DSN',)
DSN literally means "Data Source Name". So if we have no another data source, we don't need to have prefix here.
It doesn't mean that it's wrong and you should delete it right now though.
@ -18,2 +16,2 @@
self.conn.row_factory = sqlite3.Row
self.cur = self.conn.cursor()
self.log.debug('Database.__init__(DSN=\'%s\')', dsn)
self.conn = psycopg2.connect(dsn)
Let's not forget about type hints.
@ -26,2 +24,3 @@
self.cur.execute('INSERT INTO users (telegram_id) VALUES (%s)', [telegram_id])
self.conn.commit()
return self.cur.lastrowid
return self.find_user(telegram_id)
WAT
RETURNING
)@ -42,2 +40,3 @@
self.cur.execute('INSERT INTO feeds (url) VALUES (%s)', [url])
self.conn.commit()
return self.cur.lastrowid
return self.find_feed_by_url(url)
WAT
@ -122,3 +121,3 @@
return list(map(lambda x: x['telegram_id'], subscribers))
def find_feeds(self) -> list[sqlite3.Row]:
def find_feeds(self) -> list[psycopg2.extras.DictRow]:
You can import
DictRow
.@ -179,3 +177,3 @@
'CREATE TABLE IF NOT EXISTS feeds_last_items ('
' feed_id INTEGER,'
' url TEXT NOT NULL UNIQUE,'
' url TEXT NOT NULL,'
Do not forget to use GUID as the base for comparison. URL's are not reliable in our case.
@ -157,3 +158,3 @@
if self.sent_counter >= self.BATCH_LIMIT:
self.log.debug('Requests limit exceeded, sleeping for a second')
time.sleep(1)
time.sleep(2)
Why?
@ -128,3 +128,3 @@
return self.cur.fetchall()
def find_user_feeds(self, user_id: int) -> list[sqlite3.Row]:
def find_user_feeds(self, user_id: int) -> list[DictRow]:
I forgot to tell about that earlier, but it's better not to expose specifics of database implementation in public methods.
We can fix that by returning non-library-specific structure. For example
list[dict]
orlist[collections.OrderedDict]
instead oflist[psycopg2.extras.DictRow]
.@ -186,1 +182,4 @@
)
@staticmethod
def __convert_to_list_of_dicts(rows: list[DictRow]) -> list[dict]:
Convert what? Maybe
__dictrow_to_dict_list()
или__dictrow_to_list_of_dicts()
?ok
Let's merge it!
WIP: Migrate to PostgreSQLto Migrate to PostgreSQL